Properly representing the heterogeneous distribution of bone tissue material properties is a key step in constructing subject-specific finite element (FE) bone models from computed tomography (CT) data. Conventional methods represent heterogeneity by subjectively grouping bone of similar attenuation together. A new technique characterizing the level of heterogeneity with an objective metric is presented. This technique identifies the minimal level of heterogeneity needed for an accurate FE model. Subject-specific models of the distal femur and proximal tibia were used in this study. An innovative application of an image processing technique in the context of material properties modeling was introduced to facilitate an objective grouping strategy, which gathered together bone based not only on density but also on location thus capturing the natural variation of bone density seen in CT images. A fully heterogeneous model containing unique material properties for each finite element was not necessary to generate an appropriate solution. Von Mises stress, strain energy density, and nodal displacements were predicted within 5% accuracy using a simplified FE femur model containing less than half the number of bone groups of the fully heterogeneous model. Each group contained attenuations varying less than 20% from the group mean. A substantial computational time savings of 60% was gained with the application of the new technique to assign bone mechanical properties.