The Effect of Peer Review on the Quality of Data Graphs in Annals of Emergency Medicine.

STUDY OBJECTIVE

We determine how peer review affects the quality of published data graphs and how the appointment of a graphics editor affects the quality of graphs in an academic medical journal.

METHODS

We conducted an observational time-series analysis to quantify the qualities of data graphs in original manuscripts and published research articles in Annals of Emergency Medicine from 2006 to 2012. We retrospectively analyzed 3 distinct periods: before the use of a graphics editor, graph review after a manuscript's acceptance, and graph review just before the first request for revision. Raters blinded to study year scored the quality of original and published graphs using an 85-item instrument. Editorial comments about graphs were classified into 4 major and 16 minor categories.

RESULTS

We studied 60 published articles and their corresponding original submissions during each period (2006, 2009, and 2012). The number of graphs increased 31%, their median data density increased 50%, and quality (completeness [+42%], visual clarity [+64%], and special features [+66%]) increased from submission to publication in all 3 periods. Although geometric mean (0.69, 0.86, and 1.2 pieces of information/cm) and median data density (0.44, 0.70, and 1.2 pieces of information/cm) were higher in the graphics editor phases, mean data density, completeness, visual clarity, and other markers of quality did not improve or decreased with dedicated graphics editing. The majority of published graphs were bar or pie graphs (49%, 53%, and 60% in 2006, 2009, and 2012, respectively) with low data density in all 3 years.

CONCLUSION

Peer review unquestionably improved graph quality. However, data densities of most graphs barely exceeded that of printed text, and many graphs failed to present the majority of available data and did not convey those data clearly; there remains much room for improvement. The timing of graphics editor involvement appears to affect the effect of the graph review process.