Actomyosin stress fibers, one of the main components of the cell's cytoskeleton, provide mechanical stability to adherent cells by applying and transmitting tensile forces onto the extracellular matrix (ECM) at the sites of cell-ECM adhesion. While it is widely accepted that changes in spatial and temporal distribution of stress fibers affect the cell's mechanical properties, there is no quantitative knowledge on how stress fiber amount and organization directly modulate cell stiffness. We address this key open question by combining atomic force microscopy with simultaneous fluorescence imaging of living cells, and combine for the first time reliable quantitative parameters obtained from both techniques. We show that the amount of myosin and (to a lesser extent) actin assembled in stress fibers directly modulates cell stiffness in adherent mouse fibroblasts (NIH3T3). In addition, the spatial distribution of stress fibers has a second-order modulatory effect. In particular, the presence of either fibers located in the cell periphery, aligned fibers or thicker fibers gives rise to reinforced cell stiffness. Our results provide basic and significant information that will help design optimal protocols to regulate the mechanical properties of adherent cells via pharmacological interventions that alter stress fiber assembly or via micropatterning techniques that restrict stress fiber spatial organization.