Levine David M, Co Zoe, Newmark Lisa P, Groisser Alissa R, Holmgren A Jay, Haas Jennifer S, Bates David W
Division of General Internal Medicine and Primary Care, Brigham and Women's Hospital, Boston, MA USA.
Harvard Medical School, Boston, MA USA.
NPJ Digit Med. 2020 May 21;3:74. doi: 10.1038/s41746-020-0268-9. eCollection 2020.
Mobile health applications ("apps") have rapidly proliferated, yet their ability to improve outcomes for patients remains unclear. A validated tool that addresses apps' potentially important dimensions has not been available to patients and clinicians. The objective of this study was to develop and preliminarily assess a usable, valid, and open-source rating tool to objectively measure the risks and benefits of health apps. We accomplished this by using a Delphi process, where we constructed an app rating tool called THESIS that could promote informed app selection. We used a systematic process to select chronic disease apps with ≥4 stars and <4-stars and then rated them with THESIS to examine the tool's interrater reliability and internal consistency. We rated 211 apps, finding they performed fair overall (3.02 out of 5 [95% CI, 2.96-3.09]), but especially poorly for privacy/security (2.21 out of 5 [95% CI, 2.11-2.32]), interoperability (1.75 [95% CI, 1.59-1.91]), and availability in multiple languages (1.43 out of 5 [95% CI, 1.30-1.56]). Ratings using THESIS had fair interrater reliability ( = 0.3-0.6) and excellent scale reliability ( = 0.85). Correlation with traditional star ratings was low ( = 0.24), suggesting THESIS captures issues beyond general user acceptance. Preliminary testing of THESIS suggests apps that serve patients with chronic disease could perform much better, particularly in privacy/security and interoperability. THESIS warrants further testing and may guide software and policymakers to further improve app performance, so apps can more consistently improve patient outcomes.
移动健康应用程序（“应用”）迅速普及，但其改善患者治疗效果的能力仍不明确。患者和临床医生一直无法获得一种经过验证的工具来评估应用潜在的重要方面。本研究的目的是开发并初步评估一种可用、有效且开源的评级工具，以客观衡量健康应用的风险和益处。我们通过德尔菲法实现了这一目标，在此过程中构建了一个名为THESIS的应用评级工具，该工具可促进明智的应用选择。我们采用系统的流程选择评分≥4星和<4星的慢性病应用，然后用THESIS对其进行评级，以检验该工具的评分者间信度和内部一致性。我们对211个应用进行了评级，发现它们总体表现一般（5分制下得3.02分[95%置信区间，2.96 - 3.09]），但在隐私/安全方面（5分制下得2.21分[95%置信区间，2.11 - 2.32]）、互操作性（1.75分[95%置信区间，1.59 - 1.91]）以及多语言可用性方面（5分制下得1.43分[95%置信区间，1.30 - 1.56]）表现尤其不佳。使用THESIS进行的评级具有中等的评分者间信度（= 0.3 - 0.6）和出色的量表信度（= 0.85）。与传统星级评分的相关性较低（= 0.24），这表明THESIS涵盖了一般用户接受度之外的问题。THESIS的初步测试表明，为慢性病患者服务的应用可以表现得更好，尤其是在隐私/安全和互操作性方面。THESIS值得进一步测试，并可能指导软件开发者和政策制定者进一步改善应用性能，以便应用能够更持续地改善患者治疗效果。
