Levels of evidence at the Pediatric Orthopaedic Society of North America annual meetings.

BACKGROUND

Since 2003, levels of evidence have been used in the orthopaedic literature to inform the reader of the study quality and its relative significance. Our primary research question was to identify if, since their introduction, there has been an improvement in the levels of evidence of the scientific papers presented at Pediatric Orthopaedic Society of North America (POSNA) meetings.

METHODS

The abstract of every paper presented orally at the POSNA annual meeting were identified for the years 2001, 2002 (pre-2003) and 2007, 2008 (post-2003). In all, 364 abstracts were identified by an independent reviewer who then excluded cadaver, animal, and basic science studies. The 307 included abstracts were then independently blinded and randomized. Two nonepidemiologically trained pediatric orthopaedic fellows independently assigned a study type and level of evidence to each abstract based on the primary research question. The first reviewer reanalyzed 50 randomly selected abstracts. Disagreement was resolved by consensus opinion with an epidemiologically trained pediatric orthopaedic surgeon. The interobserver and intraobserver reliability was calculated for the assignment of study type and levels of evidence. Changes in the study types and levels of evidence were analyzed to compare papers presented pre-2003 and post-2003.

RESULTS

For study type and levels of evidence the interobserver reliability between the authors showed substantial agreement (kappa 0.755 and 0.647, respectively). The intraobserver reliability also showed substantial agreement (kappa 0.806 and 0.789, respectively). Comparing pre-2003 and post-2003 studies there were no significant differences between the types of study presented. An increase in the number of level III studies was identified. A decrease in level I and level IV studies was identified.

CONCLUSIONS

There has been no significant improvement in the scientific quality of studies presented at POSNA Annual Meetings as measured by levels of evidence since their introduction. We have shown substantial agreement between nonepidemiologically trained orthopaedic surgeons when categorizing abstracts using this system. Study types and levels of evidence can be difficult to assign from the current abstract format due to their brevity and structure.

LEVEL OF EVIDENCE

Not applicable.