Are two (inexperienced) heads better than one (experienced) head? Averaging house officers' prognostic judgments for critically ill patients.

Inexperienced physicians may make prognostic judgments and management decisions about acutely ill patients in the absence of supervision. We hypothesized that mathematically combining judgments of junior and senior house officers might yield aggregate judgments as good as those made by experienced critical care attending physicians. We obtained independent quantitative assessments of the likelihood of in-hospital survival for 269 sequential intensive care unit admissions from the patient's intern or resident and the critical care fellow and attending physician on duty within 24 hours of admission, and compared these judgements with mortality data. By logistic regression, the residents' and fellows' judgments added independent prognostic information to each other (likelihood ratio chi 2, 7.6; df = 1). The junior house officers' and fellows' assessments were significantly less reliable than the attending physicians' by calibration curves, and by Brier scores, 0.126 and 0.127 vs 0.119. All physicians had good discriminating ability (receiver operating characteristic areas [SE] were 0.83 [0.03], 0.85 [0.03], 0.86 [0.03], respectively). A simple average of the residents' and fellows' judgments was slightly but significantly more reliable by calibration curve and by Brier score, 0.117, and as discriminating (ROC area = 0.85, SE = 0.03) as the attending physicians' judgments. Nonmedical studies have shown that averaging independent judgments may compensate for people's tendency to make extreme estimates, and may take advantage of their complementary abilities. This first medical application of this technique suggests that this form of voting by secret ballot may prove useful for health care teams making other judgments and decisions.