The goal of this work is to design computerized image analysis techniques for automatically characterizing lung nodule subtlety in CT images. Automated subtlety estimation methods may help in computer-aided detection (CAD) assessment by quantifying dataset difficulty and facilitating comparisons among different CAD algorithms. A dataset containing 813 nodules from 499 patients was obtained from the Lung Image Database Consortium. Each nodule was evaluated by four radiologists regarding nodule subtlety using a 5-point rating scale (1: most subtle). We developed a 3D technique for segmenting lung nodules using a prespecified initial ROI. Texture and morphological features were automatically extracted from the segmented nodules and their margins. The dataset was partitioned into trainers and testers using a 1:1 ratio. An artificial neural network (ANN) was trained with average reader subtlety scores as the reference. Effective features for characterizing nodule subtlety were selected based on the training set using the ANN and a stepwise feature selection method. The performance of the classifier was evaluated using prediction probability (PK) as an agreement measure, which is considered a generalization of the area under the receiver operating characteristic curve when the reference standard is multi-level. Using an ANN classifier trained with a set of 2 features (selected from a total of 30 features), including compactness and average gray value, the test concordance between computer scores and the average reader scores was 0.789 ± 0.014. Our results show that the proposed method had strong agreement with the average of subtlety scores provided by radiologists.