Llebot Casajuana Berta, Hoogendoorn Petra, Villalobos-Quesada Maria, Pratdepàdua Bufill Carme
Fundació TIC Salut Social, Ministry of Health, Government of Catalonia, Aragó 330-332, Barcelona, 08009, Spain, 34 667162080.
National eHealth Living Lab, Public Health and Primary Care Department, Leiden University Medical Centre, Albinusdreef 2, Leiden, 2333 ZA, The Netherlands.
JMIR Mhealth Uhealth. 2025 Jun 4;13:e67858. doi: 10.2196/67858.
Health apps are increasingly being used to promote health, manage diseases, and deliver health care services. Still, there is scarce objective information regarding their quality beyond the required Conformité Européenne mark for medical apps, leading to potential risks for users. To address these challenges, several authorities have developed health app assessment frameworks. In 2017, the TIC Salut Social Foundation (FTSS) in Catalonia developed its own health app assessment framework, which has been in use since that year. The publication of CEN ISO/TS 82304-2 (abbreviated as 82304-2)-a Technical Specification for assessing health apps-and the cocreation of the Label2Enable 82304-2 handbook for certified assessment organizations provide a unique opportunity to harmonize app assessments across the European Union.
This study aimed to perform a comparative analysis of the FTSS assessment framework with 82304-2 to explore the integration of 82304-2 in Catalonia. Our broader aim was to provide this methodology for health authorities elsewhere to consider integrating 82304-2 or other evaluation frameworks.
For the comparative analysis, a mixed methods approach was used, combining a qualitative case study with a quantitative analysis of the 2 frameworks. The qualitative evaluation covered rationale for assessment, framework characteristics, governance, workflows, quality aspects, and quality requirements. For the quantitative analysis, all FTSS and 82304-2 requirements were translated into concepts and subconcepts. A scoring system identified matches of the frameworks with these subconcepts, with scores ranging from 0 (no match) to 0.5 (partial match) and 1 (full match). Integration was evaluated considering several scenarios, including adopting the Label2Enable 82304-2 handbook, adopting the 82304-2 requirements, adapting the 82304-2 requirements to local needs, and maintaining the current FTSS framework.
The main difference between the frameworks was the app usage-based assessment (FTSS) versus evidence- and app usage-based assessment (82304-2). All 120 FTSS requirements and 74 quality aspect-related 82304-2 requirements were translated into 78 concepts and 97 subconcepts. Overall, 48% (47/97) of the subconcepts were found in both frameworks, 39% (37.5/97) were specific to 82304-2, and 13% (12.5/97) were specific to FTSS. All 82304-2-specific subconcepts and thus all 82304-2 requirements were found to be relevant to FTSS. FTSS decided to integrate (adopt and adapt) all 74 82304-2 requirements. In total, 5 FTSS-specific requirements were included in the Label2Enable 82304-2 handbook, while another 4 rigor-enhancing requirements, 1 scope-expanding requirement, and 1 context-specific requirement would be assessed on top.
The comprehensive comparative analysis of the FTSS framework and 82304-2 enabled FTSS decision-making to integrate all 82304-2 quality requirements and adopt the Label2Enable 82304-2 handbook in the future. The many new and all relevant 82304-2 concepts, the rigor of the handbook, and the few remaining FTSS-specific requirements are expected to be indicative of 82304-2's potential to make harmonized, robust health app assessments common in Catalonia and elsewhere. FTSS encourages other authorities to perform a similar evaluation.
健康应用程序越来越多地被用于促进健康、管理疾病和提供医疗保健服务。然而，除了医疗应用程序所需的欧洲合格认证标志外，关于其质量的客观信息稀缺，这给用户带来了潜在风险。为应对这些挑战，多个当局已制定了健康应用程序评估框架。2017年，加泰罗尼亚的TIC Salut Social基金会（FTSS）制定了自己的健康应用程序评估框架，自当年起开始使用。CEN ISO/TS 82304-2（简称为82304-2）——一项评估健康应用程序的技术规范——的发布以及为认证评估组织共同创建的Label2Enable 82304-2手册，为在欧盟范围内统一应用程序评估提供了独特机会。
本研究旨在对FTSS评估框架与82304-2进行比较分析，以探讨82304-2在加泰罗尼亚的整合情况。我们更广泛的目标是为其他地方的卫生当局提供这种方法，以供其考虑整合82304-2或其他评估框架。
为进行比较分析，采用了混合方法，将定性案例研究与对这两个框架的定量分析相结合。定性评估涵盖评估依据、框架特征、治理、工作流程、质量方面和质量要求。对于定量分析，将所有FTSS和82304-2要求转化为概念和子概念。一个评分系统确定框架与这些子概念的匹配情况，分数范围从0（不匹配）到0.5（部分匹配）和1（完全匹配）。考虑了多种情况来评估整合情况，包括采用Label2Enable 82304-2手册、采用82304-2要求、根据当地需求调整82304-2要求以及维持当前的FTSS框架。
框架之间的主要差异在于基于应用程序使用情况的评估（FTSS）与基于证据和应用程序使用情况的评估（82304-2）。所有120项FTSS要求和74项与82304-2质量方面相关的要求被转化为78个概念和97个子概念。总体而言，两个框架中发现48%（47/97）的子概念相同，39%（37.5/97）特定于82304-2，13%（12.5/97）特定于FTSS。发现所有特定于82304-2的子概念以及因此所有82304-2要求都与FTSS相关。FTSS决定整合（采用并调整）所有74项82304-2要求。Label2Enable 82304-2手册总共纳入了5项特定于FTSS的要求，同时另外4项加强严谨性的要求、1项扩大范围的要求和1项特定于具体情况的要求将在最高层面进行评估。
对FTSS框架和82304-2的全面比较分析使FTSS能够做出决策，在未来整合所有82304-2质量要求并采用Label2Enable 82304-2手册。众多新的且所有相关的82304-2概念、手册的严谨性以及剩余的少数特定于FTSS的要求，预计将表明82304-2有潜力在加泰罗尼亚和其他地方使统一、稳健的健康应用程序评估变得普遍。FTSS鼓励其他当局进行类似评估。
