School of Computing, Ulster University, Belfast, United Kingdom.
School of Electronics, Electrical Engineering and Computer Science, Queen's University Belfast, Belfast, United Kingdom.
JMIR Mhealth Uhealth. 2023 Nov 23;11:e47043. doi: 10.2196/47043.
There are more than 350,000 digital health interventions (DHIs) in the app stores. To ensure that they are effective and safe to use, they should be assessed for compliance with best practice standards.
The objective of this paper was to examine and compare the compliance of DHIs with best practice standards and adherence to user experience (UX), professional and clinical assurance (PCA), and data privacy (DP).
We collected assessment data from 1574 DHIs using the Organisation for the Review of Care and Health Apps Baseline Review (OBR) assessment tool. As part of the assessment, each DHI received a score out of 100 for each of the abovementioned areas (ie, UX, PCA, and DP). These 3 OBR scores are combined to make up the overall ORCHA score (a proxy for quality). Inferential statistics, probability distributions, Kruskal-Wallis, Wilcoxon rank sum test, Cliff delta, and Dunn tests were used to conduct the data analysis.
We found that 57.3% (902/1574) of the DHIs had an Organisation for the Review of Care and Health Apps (ORCHA) score below the threshold of 65. The overall median OBR score (ORCHA score) for all DHIs was 61.5 (IQR 51.0-73.0) out of 100. A total of 46.2% (12/26) of DHI's health care domains had a median equal to or above the ORCHA threshold score of 65. For the 3 assessment areas (UX, DP, and PCA), DHIs scored the highest for the UX assessment 75.2 (IQR 70.0-79.6), followed by DP 65.1 (IQR 55.0-73.4) and PCA 49.6 (IQR 31.9-76.1). UX scores had the least variance (SD 13.9), while PCA scores had the most (SD 24.8). Respiratory and urology DHIs were consistently highly ranked in the National Institute for Health and Care Excellence Evidence Standards Framework tiers B and C based on their ORCHA score.
There is a high level of variability in the ORCHA scores of DHIs across different health care domains. This suggests that there is an urgent need to improve compliance with best practices in some health care areas. Possible explanations for the observed differences might include varied market maturity and commercial interests within the different health care domains. More investment to support the development of higher-quality DHIs in areas such as ophthalmology, allergy, women's health, sexual health, and dental care may be needed.
应用商店中有超过 35 万个数字健康干预措施（DHIs）。为确保它们有效且安全使用，应评估其是否符合最佳实践标准。
本文旨在检查和比较 DHIs 对最佳实践标准的遵守情况以及对用户体验（UX）、专业和临床保证（PCA）和数据隐私（DP）的遵守情况。
我们使用组织审查护理和健康应用程序基线审查（OBR）评估工具从 1574 个 DHIs 中收集评估数据。作为评估的一部分，每个 DHI 在上述每个领域（即 UX、PCA 和 DP）获得 100 分的分数。这 3 个 OBR 分数组合起来构成了整体 ORCHA 分数（代表质量）。使用推断统计学、概率分布、克鲁斯卡尔-沃利斯检验、威尔科克森秩和检验、克里夫德尔塔检验和邓恩检验进行数据分析。
我们发现，57.3%（902/1574）的 DHIs 的 ORCHA 评分低于 65 的阈值。所有 DHIs 的总体中位数 OBR 评分（ORCHA 评分）为 61.5（IQR 51.0-73.0）。共有 46.2%（12/26）的 DHI 的医疗保健领域中位数等于或高于 ORCHA 的 65 分阈值。对于 3 个评估领域（UX、DP 和 PCA），DHIs 在 UX 评估中得分最高，为 75.2（IQR 70.0-79.6），其次是 DP 65.1（IQR 55.0-73.4）和 PCA 49.6（IQR 31.9-76.1）。UX 分数的方差最小（SD 13.9），而 PCA 分数的方差最大（SD 24.8）。基于 ORCHA 评分，呼吸和泌尿科 DHIs 在国家卫生与保健卓越研究所证据标准框架的 B 级和 C 级中始终排名很高。
不同医疗保健领域的 DHIs 的 ORCHA 评分存在高度差异。这表明在某些医疗保健领域迫切需要提高对最佳实践的遵守程度。观察到的差异可能的解释包括不同医疗保健领域的市场成熟度和商业利益差异。可能需要在眼科、过敏、女性健康、性健康和牙科等领域投入更多资金来支持更高质量的 DHIs 的开发。
