Healey Elizabeth, Tan Amelia Li Min, Flint Kristen L, Ruiz Jessica L, Kohane Isaac
Program in Health Sciences and Technology, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA.
Department of Biomedical Informatics, Harvard Medical School, Boston, MA, 02115, USA.
Sci Rep. 2025 Jan 7;15(1):1143. doi: 10.1038/s41598-024-84003-0.
Continuous glucose monitors (CGM) provide valuable insights about glycemic control that aid in diabetes management. However, interpreting metrics and charts and synthesizing them into linguistic summaries is often non-trivial for patients and providers. The advent of large language models (LLMs) has enabled real-time text generation and summarization of medical data. The objective of this study was to assess the strengths and limitations of using an LLM to analyze raw CGM data and produce summaries of 14 days of data for patients with type 1 diabetes. We first evaluated the ability of GPT-4 to compute quantitative metrics specific to diabetes found in an Ambulatory Glucose Profile (AGP). Then, using two independent clinician graders, we evaluated the accuracy, completeness, safety, and suitability of qualitative descriptions produced by GPT-4 across five different CGM analysis tasks. GPT-4 performed 9 out of the 10 quantitative metrics tasks with perfect accuracy across all 10 cases. The clinician-evaluated CGM analysis tasks had good performance across measures of accuracy [lowest task mean score 8/10, highest task mean score 10/10], completeness [lowest task mean score 7.5/10, highest task mean score 10/10], and safety [lowest task mean score 9.5/10, highest task mean score 10/10]. Our work serves as a preliminary study on how generative language models can be integrated into diabetes care through data summarization and, more broadly, the potential to leverage LLMs for streamlined medical time series analysis.
连续血糖监测仪（CGM）能提供有关血糖控制的宝贵见解，有助于糖尿病管理。然而，对于患者和医疗服务提供者而言，解读指标和图表并将其综合成语言总结往往并非易事。大语言模型（LLM）的出现使得能够实时生成和总结医疗数据。本研究的目的是评估使用大语言模型分析原始CGM数据并为1型糖尿病患者生成14天数据总结的优势和局限性。我们首先评估了GPT-4计算动态血糖图谱（AGP）中特定糖尿病定量指标的能力。然后，我们使用两名独立的临床医生评分员，评估了GPT-4在五个不同CGM分析任务中生成的定性描述的准确性、完整性、安全性和适用性。GPT-4在所有10个案例中的10项定量指标任务中，有9项任务的准确率达到了完美。临床医生评估的CGM分析任务在准确性[最低任务平均得分8/10，最高任务平均得分10/10]、完整性[最低任务平均得分7.5/10，最高任务平均得分10/10]和安全性[最低任务平均得分9.5/10，最高任务平均得分10/10]方面表现良好。我们的工作是一项关于如何通过数据总结将生成式语言模型整合到糖尿病护理中的初步研究，更广泛地说，是关于利用大语言模型进行简化医疗时间序列分析的潜力的研究。
