Immune checkpoint inhibitors (ICIs) such as anti-PD-1 and anti-CTLA-4 therapy are now FDA-approved treatment options for different cancer types. However, the therapeutic efficacy of ICIs varies substantially among cancer types and patients, and only a limited proportion of cancer patients benefit clinically from ICIs. To improve the therapeutic efficacy of cancer treatments involving ICI, the mechanisms of response to ICIs and the heterogeneous pattern of immune checkpoint receptor expression need to be better understood. Here, we review recent studies on ICIs in human cancer, providing the necessary basis for the rational design of immunotherapy and for appropriate patient selection.