Centre for Public Health, Department of General Practice and Family Medicine, Medical University of Vienna, Austria.
BMC Public Health. 2011 Jul 25;11:588. doi: 10.1186/1471-2458-11-588.
A 'societal impact factor' that complements the scientific impact factor would contribute to a more comprehensive evaluation of scientific research. In order to develop a practical tool for its assessment, it is important to learn about perceptions of scientists on how to measure a societal impact factor.
This qualitative study presents the development of a practical tool to measure the societal impact of publications based on 8 focus group discussions with 24 biomedical scientists at the Medical University Vienna between May 2008 and May 2009. Topics focused on (1) features of an ideal tool, (2) criteria that should be considered in the assessment, and (3) the identification of practical pitfalls. In an iterative exercise involving the repeated application of the drafted tool to scientific papers, criteria for the assessment were refined. A small-scale exercise to evaluate the tool in terms of its comprehensibility, relevance and practicability was conducted using questionnaires for 6 external experts in leading positions of public health, and yielded acceptable results.
The tool developed consists of three quantitative dimensions, that is (1) the aim of a publication, (2) the efforts of the authors to translate their research results, and, if translation was accomplished, (3) (a) the size of the area where translation was accomplished (regional, national or international), (b) its status (preliminary versus permanent) and (c) the target group of the translation (individuals, subgroup of population, total population).
Focus group discussions with scientists suggested that the societal impact factor of a publication should consider the effect of the publication in a wide set of non-scientific areas, but also the motivation behind the publication, and efforts by the authors to translate their findings. The proposed tool provides some valuable insights for further research and practical applications in the topic area.
一个补充科学影响因子的“社会影响因子”将有助于更全面地评价科学研究。为了开发一种实用的评估工具，了解科学家对如何衡量社会影响因子的看法非常重要。
本定性研究展示了一种基于 2008 年 5 月至 2009 年 5 月在维也纳医科大学进行的 8 次焦点小组讨论，开发一种实用工具来衡量出版物的社会影响。焦点小组讨论的主题集中在（1）理想工具的特征，（2）评估中应考虑的标准，以及（3）识别实际陷阱。在一个涉及反复应用起草工具评估科学论文的迭代过程中，对评估标准进行了细化。通过向公共卫生领域的 6 位外部专家发放问卷，对工具的理解性、相关性和实用性进行了小规模评估，结果可接受。
开发的工具由三个定量维度组成，即（1）出版物的目的，（2）作者将研究结果转化的努力，如果翻译完成，则（3）（a）翻译完成的区域大小（区域、国家或国际），（b）其地位（初步还是永久）和（c）翻译的目标群体（个人、人群中的一部分、总人口）。
与科学家的焦点小组讨论表明，出版物的社会影响因子应考虑出版物在广泛的非科学领域的影响，以及出版物背后的动机，以及作者将研究结果转化的努力。所提出的工具为进一步的研究和在该主题领域的实际应用提供了一些有价值的见解。
