Grant Sean, Smart Rosanna
RAND Corporation, 1776 Main St, Santa Monica, CA 90401, USA.
HEDCO Institute for Evidence-Based Educational Practice, College of Education, University of Oregon, 1215 University of Oregon, Eugene, OR 97403-1215, USA.
MethodsX. 2024 May 9;12:102751. doi: 10.1016/j.mex.2024.102751. eCollection 2024 Jun.
We developed an expert panel approach for identifying expert views on the effectiveness and implementability of population-level policy interventions. ROMPER-the RAND/USC OPTIC Method for Policy Expert Ratings-involves an online, three-round, modified-Delphi process:•Experts rate and comment on policies according to domains of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Evidence-to-Decision framework.•To identify consensus on policy effectiveness and implementability, expert ratings are analyzed using the Inter-Percentile Range Adjusted for Symmetry (IPRAS) technique from the RAND/UCLA Appropriateness Method and visualized using a forest plot. To explain consensus, expert comments are analyzed using reflexive thematic analysis and reported following the Standards for Reporting Qualitative Research.•To provide actionable information for decisionmakers, each policy is summarized in a "Policy Profile" adapted from GRADEPro Evidence-to-Decision tables.We validated ROMPER in two studies that successfully recruited the targeted sample size, retained experts through all three rounds, and examined consensus on which policies are (not) effective and implementable. ROMPER protocols, materials, data, and code are openly available on the Open Science Framework with Creative Commons licensing for replication and reuse. ROMPER provides a validated, replicable, open access approach for eliciting expert views on both policy effectiveness and implementability-and for summarizing (lack of) consensus specifically for policymakers.
我们开发了一种专家小组方法，用于确定关于人群层面政策干预措施的有效性和可实施性的专家意见。ROMPER——兰德公司/南加州大学政策专家评级方法——涉及一个在线的、三轮的、改良的德尔菲过程：
•专家根据推荐分级评估、制定与评价（GRADE）证据到决策框架的领域对政策进行评级和评论。
•为了确定政策有效性和可实施性方面的共识，使用兰德公司/加州大学洛杉矶分校适宜性方法中的对称调整百分位数范围（IPRAS）技术分析专家评级，并使用森林图进行可视化展示。为了解释共识，使用反思性主题分析对专家评论进行分析，并按照定性研究报告标准进行报告。
•为决策者提供可操作的信息，每个政策都在从GRADEPro证据到决策表改编而来的“政策概况”中进行总结。
我们在两项研究中对ROMPER进行了验证，这两项研究成功招募到了目标样本量，并在三轮中都留住了专家，还考察了关于哪些政策有效（或无效）以及可实施（或不可实施）的共识。ROMPER的方案、材料、数据和代码在开放科学框架上公开提供，并采用知识共享许可协议以便复制和重用。ROMPER提供了一种经过验证的、可复制的、开放获取的方法，用于征求专家对政策有效性和可实施性的意见，并专门为政策制定者总结（缺乏）共识。
