Western Kenya suffers a highly endemic and also very heterogeneous epidemic of human immunodeficiency virus (HIV). Although female sex workers (FSW) and their male clients are known to be at high risk for HIV, HIV prevalence across regions in Western Kenya is not strongly correlated with the fraction of women engaged in commercial sex. An agent-based network model of HIV transmission, geographically stratified at the county level, was fit to the HIV epidemic, scale-up of interventions, and populations of FSW in Western Kenya under two assumptions about the potential mobility of FSW clients. In the first, all clients were assumed to be resident in the same geographies as their interactions with FSW. In the second, some clients were considered non-resident and engaged only in interactions with FSW, but not in longer-term non-FSW partnerships in these geographies. Under both assumptions, the model successfully reconciled disparate geographic patterns of FSW and HIV prevalence. Transmission patterns in the model suggest a greater role for FSW in local transmission when clients were resident to the counties, with 30.0% of local HIV transmissions attributable to current and former FSW and clients, compared to 21.9% when mobility of clients was included. Nonetheless, the overall epidemic drivers remained similar, with risky behavior in the general population dominating transmission in high-prevalence counties. Our modeling suggests that co-location of high-risk populations and generalized epidemics can further amplify the spread of HIV, but that large numbers of formal FSW and clients are not required to observe or mechanistically explain high HIV prevalence in the general population.