Department of Radiology and Nuclear Medicine, Amsterdam University Medical Center, Amsterdam, Netherlands.
Department of Public Health and Primary Care, Leiden University Medical Centre, Leiden, Netherlands.
J Med Internet Res. 2024 Aug 28;26:e50251. doi: 10.2196/50251.
Profound scientific evaluation of novel digital health technologies (DHTs) is key to enhance successful development and implementation. As such, we previously developed the eHealth evaluation cycle. The eHealth evaluation cycle contains 5 consecutive study phases: conceptual, development, feasibility, effectiveness, and implementation.
The aim of this study is to develop a better understanding of the daily practice of the eHealth evaluation cycle. Therefore, the objectives are to conduct a structured analysis of literature data to analyze the practice of the evaluation study phases and to determine which evaluation approaches are used in which study phase of the eHealth evaluation cycle.
We conducted a systematic literature search in PubMed including the MeSH term "telemedicine" in combination with a wide variety of evaluation approaches. Original peer-reviewed studies published in the year 2019 (pre-COVID-19 cohort) were included. Nonpatient-focused studies were excluded. Data on the following variables were extracted and systematically analyzed: journal, country, publication date, medical specialty, primary user, functionality, evaluation study phases, and evaluation approach. RStudio software was used to summarize the descriptive data and to perform statistical analyses.
We included 824 studies after 1583 titles and abstracts were screened. The majority of the evaluation studies focused on the effectiveness (impact; 304/824, 36.9%) study phase, whereas uptake (implementation; 70/824, 8.5%) received the least focus. Randomized controlled trials (RCTs; 170/899, 18.9%) were the most commonly used DHT evaluation method. Within the effectiveness (impact) study phase, RCTs were used in one-half of the studies. In the conceptual and planning phases, survey research (27/78, 35%) and interview studies (27/78, 35%) were most frequently used. The United States published the largest amount of DHT evaluation studies (304/824, 36.9%). Psychiatry and mental health (89/840, 10.6%) and cardiology (75/840, 8.9%) had the majority of studies published within the field.
We composed the first comprehensive overview of the actual practice of implementing consecutive DHT evaluation study phases. We found that the study phases of the eHealth evaluation cycle are unequally studied and most attention is paid to the effectiveness study phase. In addition, the majority of the studies used an RCT design. However, in order to successfully develop and implement novel DHTs, stimulating equal evaluation of the sequential study phases of DHTs and selecting the right evaluation approach that fits the iterative nature of technology might be of the utmost importance.
深入评估新型数字健康技术（DHT）对于促进其成功开发和实施至关重要。为此，我们之前开发了电子健康评估周期。电子健康评估周期包含 5 个连续的研究阶段：概念、开发、可行性、有效性和实施。
本研究旨在更好地了解电子健康评估周期的实际应用。因此，本研究的目的是对文献数据进行结构化分析，以分析评估研究阶段的实践，并确定在电子健康评估周期的哪个研究阶段使用哪种评估方法。
我们在 PubMed 中进行了系统的文献检索，其中包括“远程医疗”的 MeSH 术语，并结合了各种评估方法。纳入了 2019 年发表的（COVID-19 前队列）原始同行评审研究。排除了非患者为重点的研究。提取并系统分析了以下变量的数据：期刊、国家/地区、出版日期、医学专业、主要使用者、功能、评估研究阶段和评估方法。RStudio 软件用于总结描述性数据并进行统计分析。
经过对 1583 个标题和摘要进行筛选，我们纳入了 824 项研究。大多数评估研究侧重于有效性（影响；304/824，36.9%）研究阶段，而接受度（实施；70/824，8.5%）则关注较少。随机对照试验（RCT；170/899，18.9%）是最常用的 DHT 评估方法。在有效性（影响）研究阶段，RCT 在一半的研究中使用。在概念和规划阶段，调查研究（27/78，35%）和访谈研究（27/78，35%）最为常用。美国发表了数量最多的 DHT 评估研究（304/824，36.9%）。精神病学和心理健康（89/840，10.6%）和心脏病学（75/840，8.9%）发表的研究数量最多。
我们首次全面概述了连续实施 DHT 评估研究阶段的实际情况。我们发现电子健康评估周期的研究阶段研究不均衡，大多数注意力集中在有效性研究阶段。此外，大多数研究使用 RCT 设计。然而，为了成功开发和实施新型 DHT，刺激对 DHT 连续研究阶段的同等评估并选择适合技术迭代性质的正确评估方法可能至关重要。
