Centre de Recherche des Cordeliers, Sorbonne Université, INSERM, Université Paris Cité, Paris, F-75006, France.
HeKA, Inria Paris, Paris, F-75012, France.
BMC Med Inform Decis Mak. 2024 May 24;24(1):134. doi: 10.1186/s12911-024-02538-8.
There are approximately 8,000 different rare diseases that affect roughly 400 million people worldwide. Many of them suffer from delayed diagnosis. Ciliopathies are rare monogenic disorders characterized by a significant phenotypic and genetic heterogeneity that raises an important challenge for clinical diagnosis. Diagnosis support systems (DSS) applied to electronic health record (EHR) data may help identify undiagnosed patients, which is of paramount importance to improve patients' care. Our objective was to evaluate three online-accessible rare disease DSSs using phenotypes derived from EHRs for the diagnosis of ciliopathies.
Two datasets of ciliopathy cases, either proven or suspected, and two datasets of controls were used to evaluate the DSSs. Patient phenotypes were automatically extracted from their EHRs and converted to Human Phenotype Ontology terms. We tested the ability of the DSSs to diagnose cases in contrast to controls based on Orphanet ontology.
A total of 79 cases and 38 controls were selected. Performances of the DSSs on ciliopathy real world data (best DSS with area under the ROC curve = 0.72) were not as good as published performances on the test set used in the DSS development phase. None of these systems obtained results which could be described as "expert-level". Patients with multisystemic symptoms were generally easier to diagnose than patients with isolated symptoms. Diseases easily confused with ciliopathy generally affected multiple organs and had overlapping phenotypes. Four challenges need to be considered to improve the performances: to make the DSSs interoperable with EHR systems, to validate the performances in real-life settings, to deal with data quality, and to leverage methods and resources for rare and complex diseases.
Our study provides insights into the complexities of diagnosing highly heterogenous rare diseases and offers lessons derived from evaluation existing DSSs in real-world settings. These insights are not only beneficial for ciliopathy diagnosis but also hold relevance for the enhancement of DSS for various complex rare disorders, by guiding the development of more clinically relevant rare disease DSSs, that could support early diagnosis and finally make more patients eligible for treatment.
全球约有 8000 种不同的罕见疾病，影响约 4 亿人。其中许多人存在诊断延迟的问题。纤毛病是一种罕见的单基因遗传病，其表型和遗传异质性显著，这给临床诊断带来了巨大的挑战。应用于电子健康记录（EHR）数据的诊断支持系统（DSS）可以帮助识别未确诊的患者，这对于改善患者的护理至关重要。我们的目标是使用从 EHR 中提取的表型来评估三种在线获取的纤毛病 DSS，以诊断纤毛病。
使用经证实或疑似纤毛病的病例数据集和对照数据集来评估 DSS。从他们的 EHR 中自动提取患者表型，并将其转换为人类表型本体论术语。我们根据孤儿病本体论测试了 DSS 基于病例和对照进行诊断的能力。
共选择了 79 例病例和 38 例对照。在纤毛病真实世界数据上，DSS 的性能（最佳 DSS 的 ROC 曲线下面积为 0.72）不如在开发阶段测试集中的表现。这些系统没有一个能达到“专家级”的水平。有多系统症状的患者通常比只有孤立症状的患者更容易诊断。与纤毛病容易混淆的疾病通常会影响多个器官，并具有重叠的表型。要提高性能，需要考虑四个挑战：使 DSS 与 EHR 系统兼容，在现实环境中验证性能，处理数据质量，以及利用罕见和复杂疾病的方法和资源。
本研究深入了解了诊断高度异质的罕见疾病的复杂性，并从在现实环境中评估现有 DSS 中吸取了经验教训。这些见解不仅对纤毛病的诊断有帮助，而且对增强各种复杂罕见疾病的 DSS 也具有相关性，通过指导开发更具临床相关性的罕见疾病 DSS，从而支持早期诊断，最终使更多患者有资格接受治疗。
