Walter S D, Cook D J, Guyatt G H, King D, Troyan S
Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada.
Control Clin Trials. 1997 Feb;18(1):27-42. doi: 10.1016/s0197-2456(96)00131-6.
Considerable effort is often expended to adjudicate outcomes in clinical trials, but little has been written on the administration of the adjudication process and its possible impact on study results. As a case study, we describe the function and performance of an adjudication committee in a large randomized trial of two diagnostic approaches to potentially operable lung cancer. Up to five independent adjudicators independently determined two primary outcomes: tumor status at death or at final follow-up and the cause of death. Patients for whom there was any disagreement were discussed in committee until a consensus was achieved. We describe the pattern of agreement among the adjudicators and with the final consensus result. Additionally, we model the adjudication process and predict the results if a smaller committee had been used. We found that reducing the number of adjudicators from five to two or three would probably have changed the consensus outcome in less than 10% of cases. Correspondingly, the effect on the final study results (comparing primary outcomes in both randomized arms) would have been altered very little. Even using a single adjudicator would not have affected the results substantially. About 90 minutes of person-time per patient was required for activities directly related to the adjudication process, or approximately 6 months of full time work for the entire study. This level of effort could be substantially reduced by using fewer adjudicators with little impact on the results. Thus, we suggest that when high observer agreement is demonstrated or anticipated, adjudication committees should consist of no more than three members. Further work is needed to evaluate if smaller committees are adequate to detect small but important treatment effects or if they compromise validity when the level of adjudicator agreement is lower.
在临床试验中，人们常常花费大量精力来判定结果，但关于判定过程的管理及其对研究结果可能产生的影响却鲜有著述。作为一个案例研究，我们描述了一个判定委员会在一项关于两种潜在可手术肺癌诊断方法的大型随机试验中的功能和表现。多达五名独立的判定者独立确定两个主要结果：死亡时或最终随访时的肿瘤状态以及死亡原因。对于存在任何分歧的患者，委员会会进行讨论，直至达成共识。我们描述了判定者之间以及与最终共识结果的一致模式。此外，我们对判定过程进行建模，并预测如果使用规模较小的委员会会产生的结果。我们发现，将判定者人数从五名减少到两名或三名，在不到10%的病例中可能会改变共识结果。相应地，对最终研究结果（比较两个随机分组的主要结果）的影响也会非常小。即使只使用一名判定者，也不会对结果产生实质性影响。与判定过程直接相关的活动，每名患者大约需要90分钟的人工时间，或者整个研究大约需要6个月的全职工作时间。通过使用更少的判定者，在对结果影响不大的情况下，可以大幅减少这种工作量。因此，我们建议，当证明或预期有较高的观察者一致性时，判定委员会应由不超过三名成员组成。还需要进一步开展工作，以评估规模较小的委员会是否足以检测出微小但重要的治疗效果，或者当判定者一致性水平较低时，它们是否会损害有效性。
