Following the success of immune checkpoint blockers (ICBs) in different cancer types, a large number of studies are currently investigating ICBs in patients with hepatocellular carcinoma (HCC), alone or in combination with other treatments. Both nivolumab and pembrolizumab, as well as the combination of nivolumab plus ipilimumab have been granted accelerated approval by the United States Food and Drug Administration for sorafenib-pretreated patients. While nivolumab and pembrolizumab both failed to meet their primary endpoints in phase III trials, the combination of atezolizumab plus bevacizumab eventually improved overall and progression-free survival compared with sorafenib in a front-line phase III trial, and thus, will become the new standard of care in this setting. Despite this breakthrough, there are patient populations with certain underlying conditions that may not be ideal candidates for this new treatment either due to safety concerns or potential lack of efficacy. In this review, we discuss the safety of ICBs in patients with pre-existing autoimmune disease, IBD or a history of solid organ transplantation. Moreover, we summarise emerging preclinical and clinical data suggesting that ICBs may be less efficacious in patients with underlying non-alcoholic steatohepatitis or HCCs with activated Wnt/β-catenin signalling.