The composition of normative groups and diagnostic decision making: shooting ourselves in the foot.

PURPOSE

The normative group of a norm-referenced test is intended to provide a basis for interpreting test scores. However, the composition of the normative group may facilitate or impede different types of diagnostic interpretations. This article considers who should be included in a normative sample and how this decision must be made relative to the purpose for which a test is intended.

METHOD

The way in which the composition of the normative sample affects classification accuracy is demonstrated through a test review followed by a simulation study. The test review examined the descriptions of the normative group in a sample of 32 child language tests. The mean performance reported in the test manual for the sample of language impaired children was compared with the sample's norms, which either included or excluded children with language impairment. For the simulation, 2 contrasting normative procedures were modeled. The first procedure included a mixed group of representative cases (language impaired and normal cases). The second procedure excluded the language impaired cases from the norm.

RESULTS

Both the data obtained from test manuals and the data simulation based on population characteristics supported our claim that use of mixed normative groups decreases the ability to accurately identify language impairment. Tests that used mixed norms had smaller differences between the normative and language impaired groups in comparison with tests that excluded children with impairment within the normative sample. The simulation demonstrated mixed norms that lowered the group mean and increased the standard deviation, resulting in decreased classification accuracy.

CONCLUSIONS

When the purpose of testing is to identify children with impaired language skills, including children with language impairment in the normative sample can reduce identification accuracy.