Ebraheem Sara, Manzour Ayat Farouk, Elbokl Ahmed, Emara Tamer, Houssinie Moustafa El, Al-Tehewy Mahi Mahmoud
Department of Healthcare Quality, Faculty of Medicine, Ain Shams University, Cairo, Egypt.
Department of Community, Environmental and Occupational Medicine, Faculty of Medicine, Ain Shams University, Cairo, Egypt.
BMC Health Serv Res. 2025 Jul 1;25(1):868. doi: 10.1186/s12913-025-12733-6.
BACKGROUND: Performance measurement is crucial for maintaining quality of care in health systems, and so telemedicine is no exception. Key Performance Indicators (KPIs) are some of the primary tools for performance measurement. While there are many global studies on telemedicine quality, studies that are done specifically on indicators are limited. Based on this, the purpose of this study is to develop KPIs for evaluating the performance of telemedicine settings. METHODS: A consensus-based methodological study employing the Delphi method was conducted. The study was conducted in Ain Shams Virtual Hospital (AVH). It started with defining the selection criteria to be used in Delphi method. Then a review of scientific sources was carried out to enlist the potential indicators of telemedicine. An electronic modified Delphi technique was then used. The panel members were invited to rate the indicators on a 10-point Likert scale, and subsequently, the experts then reconsidered their previous voting in the light of the aggregated results. The agreement was reached as the indicator showed a 70% level of agreement. Cohen Kappa (K) was calculated to show the percentage of agreement between the two rounds. RESULTS: The multi-voting process established three selection criteria for indicators: clarity, feasibility, and importance. A review of research sources led to an initial list of 52 potential indicators, categorized into operations, clinical services, and customer satisfaction. The Delphi method was carried out in two rounds, with expert participation rates of 35% in the first round, and 80% in the second. In the end, 31 indicators achieved the 70% agreement level, with the majority falling under the operations category, while all customer satisfaction indicators were selected. Cohen's Kappa analysis showed that nearly 85% of comparisons between the two rounds demonstrated substantial agreement. CONCLUSIONS: This study identifies a thorough collection of indicators that offers a valuable tool for enhancing quality in telemedicine settings. The indicators cover important domains in telemedicine services, such as: access, utilization, system efficiency, and customer satisfaction. Future directions should include piloting these KPIs in telemedicine systems and assessing the acceptability of their application.
背景:绩效评估对于维持卫生系统的医疗质量至关重要,远程医疗也不例外。关键绩效指标(KPI)是绩效评估的一些主要工具。虽然有许多关于远程医疗质量的全球研究,但专门针对指标的研究有限。基于此,本研究的目的是制定用于评估远程医疗环境绩效的关键绩效指标。 方法:采用德尔菲法进行了一项基于共识的方法学研究。该研究在艾因夏姆斯虚拟医院(AVH)进行。首先定义了德尔菲法中使用的选择标准。然后对科学文献进行综述,以列出远程医疗的潜在指标。随后使用了电子改良德尔菲技术。邀请小组成员对指标进行10分制的李克特量表评分,随后,专家们根据汇总结果重新考虑他们之前的投票。当指标显示出70%的一致水平时达成共识。计算科恩卡帕系数(K)以显示两轮之间的一致百分比。 结果:多轮投票过程确定了指标的三个选择标准:清晰度、可行性和重要性。对研究文献的综述产生了一份包含52个潜在指标的初始清单,分为运营、临床服务和客户满意度三类。德尔菲法分两轮进行,第一轮专家参与率为35%,第二轮为80%。最终,31个指标达到了70%的一致水平,其中大多数属于运营类别,而所有客户满意度指标均被选中。科恩卡帕分析表明,两轮之间近85%的比较显示出高度一致。 结论:本研究确定了一套全面的指标,为提高远程医疗环境的质量提供了一个有价值的工具。这些指标涵盖了远程医疗服务的重要领域,如:可及性、利用率、系统效率和客户满意度。未来的方向应包括在远程医疗系统中试点这些关键绩效指标,并评估其应用的可接受性。
Health Technol Assess. 2001
Emerg Med J. 2016-4
JBI Database System Rev Implement Rep. 2016-4
Cochrane Database Syst Rev. 2018-1-16
Cochrane Database Syst Rev. 2022-5-20
Health Technol Assess. 2006-9
Health Technol Assess. 2001
Br J Dermatol. 2025-3-18
Reprod Biomed Online. 2023-10
Int J Environ Res Public Health. 2022-5-17
Curr Med Res Opin. 2022-6
J Korean Med Sci. 2022-3-7
BMC Health Serv Res. 2021-10-6