Heavy metal pollution in soils has attracted great attention worldwide in recent decades. Selecting Hangzhou as a case study location, this research proposed the synthesis application of positive matrix factorization (PMF) and GeogDetector models for quantitative analysis of pollution sources, which is the basis for subsequent soil pollution prevention and remediation. In total, 2150 surface soil samples were collected across the study area. Although the mean concentrations of As, Cd, Cr, Hg, and Pb in the soils were lower than the National Environmental Quality Standards for Soils in China, the mean contents of As and Cd were higher than their corresponding local background values by approximately 1.31 and 1.59 times, respectively, indicating that heavy metals have been enriched in topsoil. Agricultural activities, industrial activities, and soil parent materials were the main sources of heavy metal pollution in the soils, accounting for 63.4%, 19.8%, and 16.8% of the total heavy metal accumulation, respectively. Cr was derived mainly from soil parent materials (80.72%). Cd was closely associated with agricultural activities (73.68%), such as sewage irrigation and application of fertilizer. Mercury was mainly attributed to industrial activities (92.38%), such as coal mining and smelting. As was related to agricultural (57.83%) and natural (35.56%) sources, and Pb was associated with industrial (42.42%) and natural (41.83%) sources. The new synthesis models are useful for estimating the source apportionment of heavy metals in soils.