Naved Bilal A, Ravishankar Shravan, Colbert Georges E, Johnston Andrew, Slott Quintan M, Luo Yuan
Department of Preventive Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Clearstep Health, Chicago, IL, USA.
NPJ Digit Med. 2025 Jul 1;8(1):390. doi: 10.1038/s41746-025-01779-9.
US health systems receive up to 200 M monthly website visitors. Connecting patient searches to the appropriate workflow requires accurate classification. A dataset of searches on ~15 US health system websites was annotated, characterized, and used to train and evaluate a multi-label, multi-class, deep neural network. This classifier was deployed to health systems touching patients in all 50 states and compared to an LLM. The training dataset contained 504 unique classes with performance of the model in classifying searches among those classes ranging from ~0.90 to ~0.70 across metrics depending on the number of classes included. GPT-4 performed similarly if given a master list and demonstrated value in providing added coverage to augment the supervised classifier's performance. The collected data revealed characteristics of patient searches in the largest, multi-center, national study of US health systems to date.
美国医疗系统每月网站访问量高达2亿人次。将患者搜索与适当的工作流程相连接需要准确分类。对约15个美国医疗系统网站上的搜索数据集进行了注释、特征分析，并用于训练和评估一个多标签、多类别深度神经网络。该分类器被部署到覆盖美国所有50个州的医疗系统中，并与一个大型语言模型进行比较。训练数据集包含504个独特类别，模型在这些类别中对搜索进行分类的性能，根据所包含类别的数量，在各项指标上从约0.90到约0.70不等。如果给GPT-4一份主列表，它的表现类似，并在提供额外覆盖范围以增强监督分类器性能方面展现出价值。在这项迄今为止规模最大的关于美国医疗系统的多中心全国性研究中，所收集的数据揭示了患者搜索的特征。
