This study evaluates the performance of convolutional neural networks (CNNs) in risk stratifying the malignant potential of thyroid nodules alongside traditional methods such as American College of Radiology Thyroid Imaging Reporting and Data System (ACR TIRADS). The data set consisted of 651 pathology-proven thyroid nodules (500 benign, 151 malignant) from 571 patients collected at a single tertiary academic medical center. Each thyroid nodule consisted of two orthogonal views (sagittal and transverse) for a total of 1,302 grayscale images. A CNN classifier was developed to identify malignancy versus benign thyroid nodules, and a nested double cross validation scheme was applied to allow for both model parameter selection and for model accuracy evaluation. All thyroid nodules were classified according to ACR TIRADS criteria and were compared with their respective CNN-generated malignancy scores. The best performing model was the MobileNet CNN ensemble with an area under the curve of 0.86 (95% confidence interval, 0.83-0.90). Thyroid nodules within the highest and lowest CNN risk strata had malignancy rates of 81.4% and 5.9%, respectively. The rate of malignancy for ACR TIRADS ranged from 0% for TR1 nodules to 60% for TR5 nodules. Convolutional neural network malignancy scores correlated well with TIRADS levels, as malignancy scores ranged from 0.194 for TR1 nodules and 0.519 for TR5 nodules. Convolutional neural networks can be trained to generate accurate malignancy risk scores for thyroid nodules. These predictive models can aid in risk stratifying thyroid nodules alongside traditional professional guidelines such as TIRADS and can function as an adjunct tool for the radiologist when identifying those patients requiring further histopathologic workup.