We explored the association between liver metastases, tumor CD8+ T-cell count, and response in patients with melanoma or lung cancer treated with the anti-PD-1 antibody, pembrolizumab. The melanoma discovery cohort was drawn from the phase I Keynote 001 trial, whereas the melanoma validation cohort was drawn from Keynote 002, 006, and EAP trials and the non-small cell lung cancer (NSCLC) cohort from Keynote 001. Liver metastasis was associated with reduced response and shortened progression-free survival [PFS; objective response rate (ORR), 30.6%; median PFS, 5.1 months] compared with patients without liver metastasis (ORR, 56.3%; median PFS, 20.1 months) P ≤ 0.0001, and confirmed in the validation cohort (P = 0.0006). The presence of liver metastasis significantly increased the likelihood of progression (OR, 1.852; P < 0.0001). In a subset of biopsied patients (n = 62), liver metastasis was associated with reduced CD8+ T-cell density at the invasive tumor margin (liver metastasis+ group, n = 547 ± 164.8; liver metastasis- group, n = 1,441 ± 250.7; P < 0.016). A reduced response rate and shortened PFS was also observed in NSCLC patients with liver metastasis [median PFS, 1.8 months; 95% confidence interval (CI), 1.4-2.0], compared with those without liver metastasis (n = 119, median PFS, 4.0 months; 95% CI, 2.1-5.1), P = 0.0094. Thus, liver metastatic patients with melanoma or NSCLC that had been treated with pembrolizumab were associated with reduced responses and PFS, and liver metastases were associated with reduced marginal CD8+ T-cell infiltration, providing a potential mechanism for this outcome. Cancer Immunol Res; 5(5); 417-24. ©2017 AACR.