Endoplasmic reticulum aminopeptidase 2 (ERAP2) is an intracellular enzyme involved in the processing of antigenic peptides intended for presentation by major histocompatibility complex class I (MHCI) molecules. Because of its role in regulating immune responses, ERAP2 is an emerging pharmacological target. Phosphinic pseudopeptides are potent transition-state analogue inhibitors of ERAP2. Previous structure-activity studies have revealed a complex but ambiguous relationship between the occupation of putative specificity pockets and the inhibitor efficacy. To address these problems, we solved crystal structures of ERAP2 in complex with two phosphinic pseudotripeptide inhibitors. Both compounds are found in the catalytic site in a canonical orientation for transition-state analogues and utilize the S1 and S2' pockets in a similar fashion. Strikingly, their P1' side chains exhibit different orientations and make interactions with distinct shallow pockets near the ERAP2 active site. These structures suggest that S1' pocket usage in ERAP2 may be inhibitor-dependent and constitute useful starting templates for the further optimization of this class of compounds.