Assessing the Effectiveness of Automatic Speech Recognition Technology in Emergency Medicine Settings: a Comparative Study of Four AI-Powered Engines.

UNLABELLED

This study investigates the potential of advanced automatic speech recognition (ASR) technology for transcribing and recognizing medical information during patient encounters, with the aim of enabling real-time clinical documentation to alleviate clinicians' workload. While ASR holds promise, its effectiveness in noisy and dynamic medical settings, such as emergency medical services (EMS), remains underexplored. To address this, four ASR engines-Google Speech-to-Text Clinical Conversation, OpenAI Speech-to-Text, Amazon Transcribe Medical, and Azure Speech-to-Text-were evaluated using 40 EMS simulation recordings. Transcriptions were analyzed for accuracy across 23 electronic health record (EHR) categories relevant to EMS. Google Speech-to-Text Clinical Conversation showed the best overall performance, excelling in categories such as "mental state" (F1 = 1.0), "allergies" (F1 = 0.912), and "electrolytes" (F1 = 1.0). However, all engines struggled with critical EMS categories like "airway" (F1 = 0.524) and "pupils" (F1 = 0.542). These findings highlight the limitations of current ASR technologies and the need for further advancements to improve accuracy and usability in time-sensitive and high-pressure medical environments.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s41666-025-00193-w.