AABB validation study of the CDC's National Healthcare Safety Network Hemovigilance Module adverse events definitions protocol.

BACKGROUND

The utility of a hemovigilance system depends on appropriate, reproducible application of system definitions. This is even more important when submissions are not reviewed by an adjudicating body. We sought to determine how participants would code adverse reactions at institutions that had or had not received training on the application of definitions used in the CDC's National Healthcare Safety Network Hemovigilance Module (HVM).

STUDY DESIGN AND METHODS

Facilities that were (11) or were not (11) submitting adverse reaction data to the HVM reviewed 36 hypothetical cases containing elements of 37 case definitions from 12 different diagnostic groups. Respondents were required to determine the type of adverse event, if any, and assign a case definition (diagnostic probability), severity, and imputability using the January 2013 HVM Surveillance Protocol definitions. Those submitting HVM data had access to an instructional slide set prepared by CDC using similar hypothetical cases. Concordance with expert analysis was determined for the two groups of respondents.

RESULTS

The frequencies of agreement with the expert assessment were not different according to prior training exposure in any of the diagnostic groups, and results were totaled across both groups. Response accuracy varied by type of categorization (adverse event type, 72.1%; match with case definition, 76.5%; severity, 69.6%; imputablity, 64.4%) and by type of adverse event.

CONCLUSION

Despite delineated definitions, considerable variability in responses was seen, and this was not reduced by the available training. This degree of inconsistency in application of the surveillance definitions could degrade the utility of comparative reports.