de la Torre Elisenda, Montane Cristina, Rubio Olga, Sampietro-Colom Laura, Camacho-Mahamud Araceli, Grau-Corral Inmaculada
Patient Advocate, Barcelona, Spain.
Hospital Clínic de Barcelona, Villarroel 170, Barcelona, 08036, Spain.
J Med Internet Res. 2025 Aug 20;27:e71600. doi: 10.2196/71600.
Digital health technologies-including mobile applications, telemedicine platforms, artificial intelligence, and eHealth tools-are transforming health care delivery by enhancing access, personalization, and efficiency. However, traditional technology readiness levels (TRLs), while widely used to assess technological maturity, do not explicitly account for patient involvement-an essential factor in usability, acceptability, and real-world effectiveness.
This study aimed to integrate patient perspectives into the TRLs framework, specifically tailoring it for applications in digital health innovations. By developing a patient-centered dimension using a Delphi methodology, this study provides actionable insights to enhance usability, acceptance, and real-world effectiveness of digital health technologies such as mHealth apps, telemedicine platforms, and eHealth solutions.
A Delphi methodology was applied, involving 24 Spanish-speaking experts from diverse disciplines, including patient advocacy, clinical care, public health, ethics, and digital health engineering. Experts evaluated patient involvement statements across 10 TRL stages using a 6-point Likert scale. Consensus was defined a priori as ≥75% agreement and a mean score of ≥4.5. The Delphi process included 2 iterative rounds, allowing for refinement of the content despite initial consensus.
The Delphi process finally included 2 rounds, achieving a final 83.3% participation rate (20 of 24 experts). In round 1, all 10 TRL statements reached the predefined consensus threshold, with median scores ranging from 5.0 to 6.0 (83.3% to 100%) and mean scores from 4.70 (TRL2, 78.3%) to 5.25 (TRL5, 87.5%). While consensus was achieved, the presence of variability and qualitative feedback-particularly in early-stage TRLs such as TRL2 (Idea)-motivated a second round for refinement. In round 2, revised statements incorporating expert feedback were re-evaluated. Agreement increased across all TRLs, with mean scores ranging from 5.00 (TRL2, 83.3%) to 5.65 (TRL5, 94.2%). In total, 4 TRLs (TRL3, TRL4, TRL5, and TRL10) received a median of 6.0, indicating a unanimous strong agreement. Key refinements included more precise patient roles in usability testing, co-creation, clinical protocol design, and implementation monitoring. The framework also integrates patient-reported experience measures and patient-reported outcome measures in TRLs 5, 7, and 8.
The PULSO-Tech-Clinic (Patient Participation, User/Usability, Literacy, System, and Observatory) Model is the first framework to systematically embed patient perspectives within the TRLs. Although consensus was achieved in the first round, a second round allowed for methodological rigor and optimization of clarity and inclusivity. This validated model enhances alignment with real-world patient needs and supports the design, evaluation, and adoption of patient-centered digital health technologies. Further research should evaluate its adaptability in diverse health care systems.
数字健康技术，包括移动应用程序、远程医疗平台、人工智能和电子健康工具，正在通过提高可及性、个性化和效率来改变医疗服务的提供方式。然而，传统的技术就绪水平（TRLs）虽然被广泛用于评估技术成熟度，但并未明确考虑患者参与度——这是可用性、可接受性和实际效果的一个关键因素。
本研究旨在将患者视角纳入TRLs框架，特别是为数字健康创新应用量身定制该框架。通过使用德尔菲法开发以患者为中心的维度，本研究提供了可操作的见解，以提高移动健康应用程序、远程医疗平台和电子健康解决方案等数字健康技术的可用性、可接受性和实际效果。
应用德尔菲法，邀请了24位来自不同学科的讲西班牙语的专家，包括患者权益倡导、临床护理、公共卫生、伦理学和数字健康工程领域。专家们使用6点李克特量表对10个TRL阶段的患者参与陈述进行评估。事先将共识定义为≥75%的一致同意且平均得分≥4.5。德尔菲过程包括两轮迭代，尽管最初已达成共识，但仍允许对内容进行完善。
德尔菲过程最终进行了两轮，最终参与率为83.3%（24位专家中的20位）。在第一轮中，所有10个TRL陈述均达到了预先定义的共识阈值，中位数得分在5.0至6.0之间（83.3%至100%），平均得分在4.70（TRL2，78.3%）至5.25（TRL5，87.5%）之间。虽然达成了共识，但存在变异性和定性反馈，特别是在诸如TRL2（概念）等早期TRL中——这促使进行第二轮完善。在第二轮中，对纳入专家反馈的修订陈述进行了重新评估。所有TRL的一致同意率均有所提高，平均得分在5.00（TRL2，83.3%）至5.65（TRL5，94.2%）之间。总共有4个TRL（TRL3、TRL4、TRL5和TRL10）的中位数得分为6.0，表明达成了一致的强烈同意。关键的完善包括在可用性测试、共同创造、临床方案设计和实施监测中更精确地界定患者角色。该框架还在TRL 5、7和8中纳入了患者报告的体验指标和患者报告的结局指标。
PULSO-Tech-Clinic（患者参与、用户/可用性、素养、系统和观察站）模型是第一个将患者视角系统地嵌入TRLs的框架。尽管在第一轮中达成了共识，但第二轮确保了方法的严谨性以及清晰度和包容性的优化。这个经过验证的模型增强了与现实世界中患者需求的契合度，并支持以患者为中心的数字健康技术的设计、评估和采用。进一步的研究应评估其在不同医疗系统中的适应性。
