Health Research Board Neonatal Encephalopathy PhD Training Network (NEPTuNE), Dublin, Ireland.
Health Research Board - Trials Methodology Research Network (HRB-TMRN), Galway, Ireland.
Trials. 2023 Jul 19;24(1):461. doi: 10.1186/s13063-023-07388-9.
Delphi surveys are commonly used to prioritise critical outcomes in core outcome set (COS) development. This trial aims to compare a three-round (Multi-Round) Delphi (MRD) with a Real-Time Delphi (RTD) in the prioritisation of outcomes for inclusion in a COS for neonatal encephalopathy treatments and explore whether 'feedback', 'iteration', and 'initial condition' effects may occur in the two survey methods.
We recruited 269 participants (parents/caregivers, healthcare providers and researchers/academics) of which 222 were randomised to either the MRD or the RTD. We investigated the outcomes prioritised in each survey and the 'feedback', 'iteration', and 'initial condition' effects to identify differences between the two survey methods.
In the RTD, n = 92 participants (83%) fully completed the survey. In the MRD, n = 60 participants (54%) completed all three rounds. Of the 92 outcomes presented, 26 (28%) were prioritised differently between the RTD and MRD. Significantly fewer participants amended their scores when shown stakeholder responses in the RTD compared to the MRD ('feedback effect'). The 'iteration effect' analysis found most experts appeared satisfied with their initial ratings in the RTD and did not amend their scores following stakeholder response feedback. Where they did amend their scores, ratings were amended substantially, suggesting greater convergence. Variance in scores reduced with subsequent rounds of the MRD ('iteration effect'). Whilst most participants did not change their initial scores in the RTD, of those that did, later recruits tended to align their final score more closely to the group mean final score than earlier recruits (an 'initial condition' effect).
The feedback effect differed between the two Delphi methods but the magnitude of this difference was small and likely due to the large number of observations rather than because of a meaningfully large difference. It did not appear to be advantageous to require participants to engage in three rounds of a survey due to the low change in scores. Larger drop-out through successive rounds in the MRD, together with a lesser convergence of scores and longer time to completion, indicate considerable benefits of the RTD approach.
NCT04471103. Registered on 14 July 2020.
德尔菲调查常用于在核心结局集（COS）开发中对关键结局进行优先级排序。本试验旨在比较三轮（多轮）德尔菲（MRD）与实时德尔菲（RTD）在新生儿脑病治疗 COS 纳入结局的优先级排序中的作用，并探讨这两种调查方法是否会出现“反馈”、“迭代”和“初始条件”效应。
我们招募了 269 名参与者（父母/照顾者、医疗保健提供者和研究人员/学者），其中 222 名被随机分配到 MRD 或 RTD。我们调查了每个调查中优先考虑的结果以及“反馈”、“迭代”和“初始条件”效应，以确定这两种调查方法之间的差异。
在 RTD 中，n=92 名参与者（83%）完整完成了调查。在 MRD 中，n=60 名参与者（54%）完成了所有三轮。在提出的 92 个结果中，26 个（28%）在 RTD 和 MRD 之间的优先级不同。与 MRD 相比，在 RTD 中显示利益相关者的反馈时，参与者修改分数的人数明显减少（“反馈效应”）。“迭代效应”分析发现，大多数专家在 RTD 中对其初始评分感到满意，并且在收到利益相关者反馈后没有修改评分。在修改分数的情况下，评分有很大的修改，表明有更大的收敛性。MRD 的后续轮次中，评分的方差减少（“迭代效应”）。尽管大多数参与者在 RTD 中没有改变其初始分数，但其中一些后来的参与者比早期的参与者更倾向于将他们的最终分数与组平均最终分数更紧密地对齐（“初始条件”效应）。
两种 Delphi 方法的反馈效应不同，但这种差异的幅度很小，可能是由于观察值数量很大，而不是因为有明显较大的差异。由于分数变化不大，因此要求参与者进行三轮调查并没有优势。MRD 中连续轮次的参与者大量流失，加上评分收敛性较低和完成时间较长，表明 RTD 方法具有明显的优势。
NCT04471103。于 2020 年 7 月 14 日注册。
