Comparing alternative and traditional dissemination metrics in medical education.

CONTEXT

The impact of academic scholarship has traditionally been measured using citation-based metrics. However, citations may not be the only measure of impact. In recent years, other platforms (e.g. Twitter) have provided new tools for promoting scholarship to both academic and non-academic audiences. Alternative metrics (altmetrics) can capture non-traditional dissemination data such as attention generated on social media platforms.

OBJECTIVES

The aims of this exploratory study were to characterise the relationships among altmetrics, access counts and citations in an international and pre-eminent medical education journal, and to clarify the roles of these metrics in assessing the impact of medical education academic scholarship.

METHODS

A database study was performed (September 2015) for all papers published in Medical Education in 2012 (n = 236) and 2013 (n = 246). Citation, altmetric and access (HTML views and PDF downloads) data were obtained from Scopus, the Altmetric Bookmarklet tool and the journal Medical Education, respectively. Pearson coefficients (r-values) between metrics of interest were then determined.

RESULTS

Twitter and Mendeley (an academic bibliography tool) were the only altmetric-tracked platforms frequently (> 50%) utilised in the dissemination of articles. Altmetric scores (composite measures of all online attention) were driven by Twitter mentions. For short and full-length articles in 2012 and 2013, both access counts and citation counts were most strongly correlated with one another, as well as with Mendeley downloads. By comparison, Twitter metrics and altmetric scores demonstrated weak to moderate correlations with both access and citation counts.

CONCLUSIONS

Whereas most altmetrics showed limited correlations with readership (access counts) and impact (citations), Mendeley downloads correlated strongly with both readership and impact indices for articles published in the journal Medical Education and may therefore have potential use that is complementary to that of citations in assessment of the impact of medical education scholarship.