Laboratory of Translational Medicine and Decision Science, TransLab Research Group, Department of Medical Sciences, Faculty of Medicine, University of Girona, 17071 Girona, Catalonia, Spain.
Soc Sci Med. 2012 Aug;75(3):505-10. doi: 10.1016/j.socscimed.2012.03.044. Epub 2012 Apr 27.
Most ex-post evaluations of research funding programs are based on bibliometric methods and, although this approach has been widely used, it only examines one facet of the project's impact, that is, scientific productivity. More comprehensive models of payback assessment of research activities are designed for large-scale projects with extensive funding. The purpose of this study was to design and implement a methodology for the ex-post evaluation of small-scale projects that would take into account both the fulfillment of projects' stated objectives as well as other wider benefits to society as payback measures. We used a two-phase ex-post approach to appraise impact for 173 small-scale projects funded in 2007 and 2008 by a Spanish network center for research in epidemiology and public health. In the internal phase we used a questionnaire to query the principal investigator (PI) on the outcomes as well as actual and potential impact of each project; in the external phase we sent a second questionnaire to external reviewers with the aim of assessing (by peer-review) the performance of each individual project. Overall, 43% of the projects were rated as having completed their objectives "totally", and 40% "considerably". The research activities funded were reported by PIs as socially beneficial their greatest impact being on research capacity (50% of payback to society) and on knowledge translation (above 11%). The method proposed showed a good discriminating ability that makes it possible to measure, reliably, the extent to which a project's objectives were met as well as the degree to which the project contributed to enhance the group's scientific performance and of its social payback.
大多数研究资助计划的事后评估都是基于文献计量方法的，尽管这种方法已经被广泛应用，但它只考察了项目影响的一个方面，即科学生产力。更全面的研究活动回报评估模型是为具有广泛资金的大规模项目设计的。本研究旨在设计和实施一种小规模项目的事后评估方法，该方法将考虑项目既定目标的实现以及其他更广泛的社会效益作为回报措施。我们使用了两阶段的事后评估方法来评估 2007 年和 2008 年由西班牙一个流行病学和公共卫生研究网络中心资助的 173 个小规模项目的影响。在内部阶段，我们使用问卷询问主要研究者（PI）关于每个项目的结果以及实际和潜在影响；在外部阶段，我们向外部评审员发送了第二份问卷，目的是通过同行评议评估每个项目的绩效。总体而言，43%的项目被评为“完全”完成了目标，40%的项目被评为“相当”完成了目标。PI 报告称，所资助的研究活动具有社会效益，其最大的影响是研究能力（对社会的回报的 50%）和知识转化（超过 11%）。所提出的方法显示出良好的区分能力，能够可靠地衡量项目目标的实现程度以及项目对增强团队科学绩效和社会效益的贡献程度。
