Rosas Scott R, Kane Mary
Concept Systems, Inc., 136 East State Street, Ithaca, NY 14850, USA.
Eval Program Plann. 2012 May;35(2):236-45. doi: 10.1016/j.evalprogplan.2011.10.003. Epub 2011 Oct 12.
The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed.
在过去20年里，概念图在研究和评估中的应用急剧增加。学术、组织和社区环境中的研究人员已成功应用概念图，但缺乏对各项研究进行系统分析以确定方法合理的研究的特征。目前缺少可指导未来研究的定量特征以及质量和严谨性的评估。为填补这一空白，我们对69项概念图研究进行了汇总分析，以描述各研究阶段的特征，生成有效性和可靠性的具体指标，并研究选定的研究特征与质量指标之间的关系。汇总并定量总结了各项研究的特征和评估结果，描述了每项特征的分布、变异和参数。此外，还研究了概念图数据收集在特征和评估方面的差异。总体而言，结果表明概念图具有很强的内部表征有效性以及非常强的分类和评分可靠性评估。尽管不同数据收集模式下的参与率和任务完成率存在差异，但有效性和可靠性一直很高。本文讨论了这些发现作为评估未来概念图研究质量和严谨性的实用参考的意义。
