is Professor and Associate Dean for Evaluation and Educational Research, the George Washington University School of Medicine and Health Sciences, and Deputy Editor, Journal of Graduate Medical Education.
is Vice President of Accreditation Standards, Accreditation Council for Graduate Medical Education (ACGME).
J Grad Med Educ. 2023 Feb;15(1):67-73. doi: 10.4300/JGME-D-22-00437.1.
Since 2003, the Accreditation Council for Graduate Medical Education (ACGME) has surveyed residents and fellows in its accredited programs. The Resident/Fellow Survey is a screening and compliance tool that programs can also use for continuous quality improvement. However, stakeholders have reported potential problems with the survey's overall quality and credibility.
To redesign the 2006 Resident/Fellow Survey using expert reviews and cognitive interviews.
In 2018-2019, the ACGME redesigned the Resident/Fellow Survey using an iterative validation process: expert reviews (evidence based on content) and cognitive interviews (evidence based on response processes). Expert reviews were conducted by a survey design firm and an ACGME Task Force; cognitive interviews were conducted with a diverse set of 27 residents and fellows.
Expert reviews resulted in 21 new survey items to address the ACGME's updated accreditation requirements; these reviews also led to improvements that align the survey items with evidence-informed standards. After these changes were made, cognitive interviews resulted in additional revisions to sensitive items, item order, and response option order, labels, and conceptual spacing. In all, cognitive interviews led to 11 item deletions and other improvements designed to reduce response error.
Expert reviews and cognitive interviews were used to redesign the Resident/Fellow Survey. The content of the redesigned survey aligns with the updated accreditation requirements and items are written in accordance with evidence-informed standards. Additionally, cognitive interviews resulted in revisions to the survey that seem to improve respondents' comprehension and willingness to respond to individual survey items.
自 2003 年以来，美国毕业后医学教育认证委员会（ACGME）一直在其认可的项目中对住院医师和研究员进行调查。住院医师/研究员调查是一种筛选和合规工具，项目也可以用于持续质量改进。然而，利益相关者报告称，该调查的整体质量和可信度存在潜在问题。
使用专家评审和认知访谈重新设计 2006 年住院医师/研究员调查。
在 2018-2019 年，ACGME 使用迭代验证过程重新设计了住院医师/研究员调查：专家评审（基于内容的证据）和认知访谈（基于反应过程的证据）。专家评审由一家调查设计公司和 ACGME 工作组进行；认知访谈由一组 27 名不同背景的住院医师和研究员进行。
专家评审导致 21 项新的调查项目，以满足 ACGME 更新的认证要求；这些评审还改进了调查项目与循证标准的一致性。在做出这些改变之后，认知访谈导致对敏感项目、项目顺序、和响应选项顺序、标签和概念间距进行了额外的修订。总之，认知访谈导致 11 项删除和其他旨在减少响应错误的改进。
使用专家评审和认知访谈重新设计了住院医师/研究员调查。重新设计的调查内容与更新的认证要求一致，并且项目是按照循证标准编写的。此外，认知访谈对调查进行了修订，似乎提高了受访者对个别调查项目的理解和愿意回应的程度。
