In previous research, we have developed a computer-aided detection (CAD) system designed to detect masses in mammograms. The previous version of our system employed a simple but imprecise method to localize the masses. In this research, we present a more robust segmentation routine for use with mammographic masses. Our hypothesis is that by more accurately describing the morphology of the masses, we can improve the CAD system's ability to distinguish masses from other mammographic structures. To test this hypothesis, we incorporated the new segmentation routine into our CAD system and examined the change in performance. The developed iterative, linear segmentation routine is a gray level-based procedure. Using the identified regions from the previous CAD system as the initial seeds, the new segmentation algorithm refines the suspicious mass borders by making estimates of the interior and exterior pixels. These estimates are then passed to a linear discriminant, which determines the optimal threshold between the interior and exterior pixels. After applying the threshold and identifying the object's outline, two constraints on the border are applied to reduce the influence of background noise. After the border is constrained, the process repeats until a stopping criterion is reached. The segmentation routine was tested on a study database of 183 mammographic images extracted from the Digital Database for Screening Mammography. Eighty-three of the images contained 50 malignant and 50 benign masses; 100 images contained no masses. The previously developed CAD system was used to locate a set of suspicious regions of interest (ROIs) within the images. To assess the performance of the segmentation algorithm, a set of 20 features was measured from the suspicious regions before and after the application of the developed segmentation routine. Receiver operating characteristic (ROC) analysis was employed on the ROIs to examine the discriminatory capabilities of each individual feature before and after the segmentation routine. A statistically significant performance increase was found in many of the individual features, particularly those describing the mass borders. To examine how the incorporation of the segmentation routine affected the performance of the overall CAD system, free-response ROC (FROC) analysis was employed. When considering only malignant masses, the FROC performance of the system with the segmentation routine appeared better than the previous system. When detecting 90% of the malignant masses, the previous system achieved 4.9 false positives per image (FPpI) compared to the post-segmentation system's 4.2 FPpI. At 80% sensitivity, the respective FPpI were 3.5 and 1.6.