HIV epidemic control requires improving access and uptake of HIV services by key populations (KPs). In Zambia, the behaviors of female sex workers (FSWs), men who have sex with men (MSM), and people of who use drugs (PWUD) are criminalized, and little information exists about their HIV/STI service use. Using a quality of care (QOC) framework, we compared barriers to and opportunities for HIV/STI service access and uptake among the three KPs. We conducted in-depth interviews and focus group discussions with 314 KP members between July 2013 and September 2015 in eight districts. Poorer QOC was received at public health facilities compared to private, NGOs and traditional healers. Stigma and discrimination, confidentiality, and legal prosecution were barriers to service use and more salient among MSM than FSWs and PWUD. Invasive facility policies were barriers and more prominent among FSWs than MSM and PWUD. Service unavailability was of equally high salience among MSM and PWUD than FSWs. Comfort in the clinic and perceived treatment effectiveness were facilitators for all three KPs. The health care experiences of KPs are not monolithic; HIV/STI service improvement strategies should address the concerns and be tailored to the needs of each key population.