Wang Ariel, Seeley Anna E, Sydes Matthew R, Jones Nicholas, de Lusignan Simon, Hobbs Fd Richard, McManus Richard J, Williams Marney, Sheppard James P
Nuffield Department of Primary Care Health Sciences, University of Oxford, Radcliffe Primary Care Building, Radcliffe Observatory Quarter, Oxford, OX2 6GG, UK.
Data for R&D, Transformation Directorate, NHS England, London, UK.
BMC Med Res Methodol. 2025 Jun 7;25(1):156. doi: 10.1186/s12874-025-02606-1.
Whilst interest in efficient trial design has grown with the use of electronic health records (EHRs) to collect trial outcomes, practical challenges remain. Commonly raised concerns often revolve around data availability, data quality and issues with data validation. This study aimed to assess the agreement between data collected on clinical trial participants from different sources to provide empirical evidence on the utility of EHRs for follow-up in randomised controlled trials (RCTs).
This retrospective, participant-level data utility comparison study was undertaken using data collected as part of a UK primary care-based, randomised controlled trial (OPTiMISE). The primary outcome measure was the recording of all-cause hospitalisation or mortality within 3 years post-randomisation and was assessed across (1) Coded primary care data; (2) Coded-plus-free-text primary care data; and (3) Coded secondary care and mortality data. Agreement levels across data sources were assessed using Fleiss' Kappa (K). Kappa statistics were interpreted using an established framework, categorising agreement strength as follows: <0 (poor), 0.00-0.20 (slight), 0.21-0.40 (fair), 0.41-0.60 (moderate), 0.61-0.80 (substantial), and 0.81-1.00 (almost perfect) agreement. The impact of using different data sources to determine trial outcomes was assessed by replicating the trial's original analyses.
Almost perfect agreement was observed for mortality outcome across the three data sources (K = 0.94, 95%CI 0.91-0.98). Fair agreement (weak consistency) was observed for hospitalisation outcomes, including all-cause hospitalisation or mortality (K = 0.35, 95%CI 0.28-0.42), emergency hospitalisation (K = 0.39, 95%CI 0.33-0.46), and hospitalisation or mortality due to cardiovascular disease (K = 0.32, 95%CI 0.19-0.45). The overall trial results remained consistent across data sources for the primary outcome, albeit with varying precision.
Significant discrepancies according to data sources were observed in recording of secondary care outcomes. Investigators should be cautious when choosing which data source(s) to use to measure outcomes in trials. Future work on linking participant-level data across healthcare settings should consider the variations in diagnostic coding practices. Standardised definitions for outcome measures when using healthcare systems data and using data from different data sources for cross-checking and verification should be encouraged.
随着利用电子健康记录（EHRs）收集试验结果，人们对高效试验设计的兴趣日益浓厚，但实际挑战依然存在。常见的担忧往往围绕数据可用性、数据质量以及数据验证问题。本研究旨在评估从不同来源收集的关于临床试验参与者的数据之间的一致性，以提供关于EHRs在随机对照试验（RCTs）随访中效用的实证证据。
本回顾性、参与者层面的数据效用比较研究使用了作为一项基于英国初级保健的随机对照试验（OPTiMISE）一部分收集的数据。主要结局指标是随机分组后3年内全因住院或死亡的记录，并在以下方面进行评估：（1）编码的初级保健数据；（2）编码加自由文本的初级保健数据；以及（3）编码的二级保健和死亡数据。使用Fleiss' Kappa（K）评估各数据源之间的一致性水平。Kappa统计量采用既定框架进行解释，将一致性强度分类如下：<0（差）、0.00 - 0.20（轻微）、0.21 - 0.40（一般）、0.41 - 0.60（中等）、0.61 - 0.80（实质性）和0.81 - 1.00（几乎完美）一致。通过重复试验的原始分析，评估使用不同数据源确定试验结果的影响。
在三个数据源中观察到死亡结局几乎完美一致（K = 0.94，95%CI 0.91 - 0.98）。对于住院结局，包括全因住院或死亡（K = 0.35，95%CI 0.28 - 0.42）、急诊住院（K = 0.39，95%CI 0.33 - 0.46）以及心血管疾病导致的住院或死亡（K = 0.32，95%CI 0.19 - 0.45），观察到一般一致（一致性较弱）。尽管精度有所不同，但主要结局的总体试验结果在各数据源之间保持一致。
在二级保健结局记录方面，根据数据源观察到显著差异。研究人员在选择用于衡量试验结局的数据源时应谨慎。未来关于跨医疗环境链接参与者层面数据的工作应考虑诊断编码实践的差异。应鼓励在使用医疗系统数据时对结局指标采用标准化定义，并使用来自不同数据源的数据进行交叉核对和验证。
