Babrak Lmar Marie, Smakaj Erand, Agac Teyfik, Asprion Petra Maria, Grimberg Frank, der Werf Daan Van, van Ginkel Erwin Willem, Tosoni Deniz David, Clay Ieuan, Degen Markus, Brodbeck Dominique, Natali Eriberto Noel, Schkommodau Erik, Miho Enkelejda
University of Applied Sciences and Arts Northwestern Switzerland, School of Life Sciences, Muttenz, Switzerland.
Fachhochschule Nordwestschweiz University of Applied Sciences and Arts Northwestern Switzerland, School of Business, Olten, Switzerland.
JMIR Form Res. 2022 Oct 18;6(10):e29920. doi: 10.2196/29920.
Digital technologies are transforming the health care system. A large part of information is generated as real-world data (RWD). Data from electronic health records and digital biomarkers have the potential to reveal associations between the benefits and adverse events of medicines, establish new patient-stratification principles, expose unknown disease correlations, and inform on preventive measures. The impact for health care payers and providers, the biopharmaceutical industry, and governments is massive in terms of health outcomes, quality of care, and cost. However, a framework to assess the preliminary quality of RWD is missing, thus hindering the conduct of population-based observational studies to support regulatory decision-making and real-world evidence.
To address the need to qualify RWD, we aimed to build a web application as a tool to translate characterization of some quality parameters of RWD into a metric and propose a standard framework for evaluating the quality of the RWD.
The RWD-Cockpit systematically scores data sets based on proposed quality metrics and customizable variables chosen by the user. Sleep RWD generated de novo and publicly available data sets were used to validate the usability and applicability of the web application. The RWD quality score is based on the evaluation of 7 variables: manageability specifies access and publication status; complexity defines univariate, multivariate, and longitudinal data; sample size indicates the size of the sample or samples; privacy and liability stipulates privacy rules; accessibility specifies how the data set can be accessed and to what granularity; periodicity specifies how often the data set is updated; and standardization specifies whether the data set adheres to any specific technical or metadata standard. These variables are associated with several descriptors that define specific characteristics of the data set.
To address the need to qualify RWD, we built the RWD-Cockpit web application, which proposes a framework and applies a common standard for a preliminary evaluation of RWD quality across data sets-molecular, phenotypical, and social-and proposes a standard that can be further personalized by the community retaining an internal standard. Applied to 2 different case studies-de novo-generated sleep data and publicly available data sets-the RWD-Cockpit could identify and provide researchers with variables that might increase quality.
The results from the application of the framework of RWD metrics implemented in the RWD-Cockpit application suggests that multiple data sets can be preliminarily evaluated in terms of quality using the proposed metrics. The output scores-quality identifiers-provide a first quality assessment for the use of RWD. Although extensive challenges remain to be addressed to set RWD quality standards, our proposal can serve as an initial blueprint for community efforts in the characterization of RWD quality for regulated settings.
数字技术正在改变医疗保健系统。很大一部分信息以真实世界数据（RWD）的形式产生。来自电子健康记录和数字生物标志物的数据有可能揭示药物的益处与不良事件之间的关联，确立新的患者分层原则，揭示未知的疾病相关性，并为预防措施提供依据。对医疗保健支付方和提供者、生物制药行业以及政府而言，在健康结果、护理质量和成本方面，其影响是巨大的。然而，目前缺少一个评估RWD初步质量的框架，这阻碍了开展基于人群的观察性研究以支持监管决策和真实世界证据。
为满足对RWD进行质量评估的需求，我们旨在构建一个网络应用程序，作为将RWD某些质量参数的特征转化为一种度量标准的工具，并提出一个评估RWD质量的标准框架。
RWD-Cockpit根据提议的质量指标和用户选择的可定制变量对数据集进行系统评分。使用新生成的睡眠RWD和公开可用的数据集来验证该网络应用程序的可用性和适用性。RWD质量评分基于对7个变量的评估：可管理性规定了访问和发布状态；复杂性定义了单变量、多变量和纵向数据；样本量表明样本或多个样本的大小；隐私和责任规定了隐私规则；可访问性规定了如何访问数据集以及访问的粒度；周期性规定了数据集更新的频率；标准化规定了数据集是否符合任何特定的技术或元数据标准。这些变量与多个描述符相关联，这些描述符定义了数据集的特定特征。
为满足对RWD进行质量评估的需求，我们构建了RWD-Cockpit网络应用程序，它提出了一个框架，并应用一个通用标准对跨数据集（分子、表型和社会数据集）的RWD质量进行初步评估，并提出了一个可由社区进一步个性化定制的标准，同时保留内部标准。应用于2个不同的案例研究——新生成的睡眠数据和公开可用的数据集——RWD-Cockpit能够识别并为研究人员提供可能提高质量的变量。
在RWD-Cockpit应用程序中实施的RWD指标框架的应用结果表明，可以使用提议的指标对多个数据集的质量进行初步评估。输出分数——质量标识符——为RWD的使用提供了首次质量评估。尽管在设定RWD质量标准方面仍有诸多挑战有待解决，但我们的提议可为社区在规范环境中表征RWD质量的努力提供一个初步蓝图。
