The Problem of Aggregating Validators for Psychiatric Disorders.

Since the time of the Diagnostic and Statistical Manual of Mental Disorders, 3rd Edition, evidence for the validity of psychiatric disorders has been expressed in the form of validators, which are instances of particular kinds of evidence. There has never been an explicit discussion of how the validators should be aggregated to come to an overall conclusion about the strength of the evidence for a psychiatric category. We include both the challenges of aggregating validators of the same type and the challenges of aggregating different types of validators. We consider five different alternatives: informal aggregation, weighted informal aggregation (simple evidence hierarchy), formal aggregation, underdetermination, and inclusion of values. Each of the alternatives has different implications. We suggest that, going forward, aggregation of validators should be more explicit, maximizing rigor and reproducibility.