Measuring performance in chest radiography.

PURPOSE

To use a standardized set of chest radiographs to quantify interobserver differences and to provide a basis for comparing the diagnostic performance of physicians.

MATERIALS AND METHODS

A standardized set of 60 chest radiographs was presented to 162 study participants. Each participant reviewed the radiographs and recorded his or her diagnostic impression by using a fixed five-point scale. These response data were used to generate receiver operating characteristic curves and to establish performance benchmarks. The variations in performance were tested for statistical significance.

RESULTS

Significant interobserver variability was identified during these assessments. The composite group of board-certified radiologists demonstrated performance superior to that of the radiology residents and nonradiologist physicians.

CONCLUSION

By using a receiver operating characteristic approach and a standardized set of chest radiographs, observer accuracy and variability are easily quantified. This approach provides a basis for comparing the diagnostic performance of physicians. When value is measured as a diminution in uncertainty, board-certified radiologists contribute substantial value to the diagnostic imaging system.