Windecker Daniel, Baj Giovanni, Shiri Isaac, Kazaj Pooya Mohammadi, Kaesmacher Johannes, Gräni Christoph, Siontis George C M
Department of Diagnostic and Interventional Neuroradiology, University of Bern, Bern, Switzerland.
Department of Cardiology, Bern University Hospital, Inselspital, University of Bern, Bern, Switzerland.
JAMA Netw Open. 2025 Apr 1;8(4):e258052. doi: 10.1001/jamanetworkopen.2025.8052.
The primary objective of any newly developed medical device using artificial intelligence (AI) is to ensure its safe and effective use in broader clinical practice.
To evaluate key characteristics of AI-enabled medical devices approved by the US Food and Drug Administration (FDA) that are relevant to their clinical generalizability and are reported in the public domain.
DESIGN, SETTING, AND PARTICIPANTS: This cross-sectional study collected information on all AI-enabled medical devices that received FDA approval and were listed on the FDA website as of August 31, 2024.
For each AI-enabled medical device, detailed information and key characteristics relevant for the generalizability of the devices at the time of approval were summarized, specifically examining clinical evaluation aspects, such as the presence and design of clinical performance studies, availability of discriminatory performance metrics, and age- and sex-specific data.
In total, 903 FDA-approved AI-enabled medical devices were analyzed, most of which became available in the last decade. The devices primarily related to the specialties of radiology (692 devices [76.6.%]), cardiovascular medicine (91 devices [10.1%]), and neurology (29 devices [3.2%]). Most devices were software only (664 devices [73.5%]), and only 6 devices (0.7%) were implantable. Detailed descriptions of development were absent from most publicly provided summaries. Clinical performance studies were reported for 505 devices (55.9%), while 218 devices (24.1%) explicitly stated no performance studies were conducted. Retrospective study designs were most common (193 studies [38.2%]), with only 41 studies (8.1%) being prospective and 12 studies (2.4%) randomized. Discriminatory performance metrics were reported in 200 of the available summaries (sensitivity: 183 devices [36.2%]; specificity: 176 devices [34.9%]; area under the curve: 82 devices [16.2%]). Among clinical studies, less than one-third provided sex-specific data (145 studies [28.7%]), and only 117 studies (23.2%) addressed age-related subgroups.
In this cross-sectional study, clinical performance studies at the time of approval were reported for approximately half of AI-enabled medical devices, yet the information was often insufficient for a comprehensive assessment of their clinical generalizability, emphasizing the need for ongoing monitoring and regular re-evaluation to identify and address unexpected performance changes during broader use.
任何新开发的使用人工智能（AI）的医疗设备的主要目标是确保其在更广泛的临床实践中安全有效地使用。
评估美国食品药品监督管理局（FDA）批准的具备人工智能的医疗设备的关键特征，这些特征与它们的临床可推广性相关且在公共领域有报道。
设计、设置和参与者：这项横断面研究收集了截至2024年8月31日已获得FDA批准并列在FDA网站上的所有具备人工智能的医疗设备的信息。
对于每台具备人工智能的医疗设备，总结了批准时与设备可推广性相关的详细信息和关键特征，特别考察了临床评估方面，如临床性能研究的存在情况和设计、鉴别性能指标的可用性以及年龄和性别特异性数据。
总共分析了903台获得FDA批准的具备人工智能的医疗设备，其中大多数是在过去十年中上市的。这些设备主要涉及放射学专业（692台设备[76.6%]）、心血管医学专业（91台设备[10.1%]）和神经学专业（29台设备[3.2%]）。大多数设备仅为软件（664台设备[73.5%]），只有6台设备（0.7%）是可植入的。大多数公开提供的总结中缺少对开发的详细描述。505台设备（55.9%）报告了临床性能研究，而218台设备（24.1%）明确表示未进行性能研究。回顾性研究设计最为常见（193项研究[38.2%]），只有41项研究（8.1%）是前瞻性研究，12项研究（2.4%）是随机研究。在可用的总结中，200份报告了鉴别性能指标（敏感性：183台设备[36.2%]；特异性：176台设备[34.9%]；曲线下面积：82台设备[16.2%]）。在临床研究中，不到三分之一提供了性别特异性数据（145项研究[28.7%]），只有117项研究（23.2%）涉及年龄相关亚组。
在这项横断面研究中，约一半的具备人工智能的医疗设备在批准时报告了临床性能研究，但这些信息往往不足以全面评估其临床可推广性，强调需要持续监测和定期重新评估，以识别和解决在更广泛使用过程中出现的意外性能变化。
