Department of Mechanical and Industrial Engineering, Northeastern University, 360 Huntington Avenue, Boston, MA, USA.
Robert H. Smith School of Business, University of Maryland, College Park, MD, USA.
Drugs. 2021 Mar;81(4):471-482. doi: 10.1007/s40265-020-01435-4.
Comorbid chronic conditions are common among people with type 2 diabetes. We developed an artificial intelligence algorithm, based on reinforcement learning (RL), for personalized diabetes and multimorbidity management, with strong potential to improve health outcomes relative to current clinical practice.
We modeled glycemia, blood pressure, and cardiovascular disease (CVD) risk as health outcomes, using a retrospective cohort of 16,665 patients with type 2 diabetes from New York University Langone Health ambulatory care electronic health records in 2009-2017. We trained an RL prescription algorithm that recommends a treatment regimen optimizing patients' cumulative health outcomes using their individual characteristics and medical history at each encounter. The RL recommendations were evaluated on an independent subset of patients.
The single-outcome optimization RL algorithms, RL-glycemia, RL-blood pressure, and RL-CVD, recommended consistent prescriptions as that observed by clinicians in 86.1%, 82.9%, and 98.4% of the encounters, respectively. For patient encounters in which the RL recommendations differed from the clinician prescriptions, significantly fewer encounters showed uncontrolled glycemia (A1c > 8% in 35% of encounters), uncontrolled hypertension (blood pressure > 140 mmHg in 16% of encounters), and high CVD risk (risk > 20% in 25% of encounters) under RL algorithms compared with those observed under clinicians (43%, 27%, and 31% of encounters, respectively; all p < 0.001).
A personalized RL prescriptive framework for type 2 diabetes yielded high concordance with clinicians' prescriptions, and substantial improvements in glycemia, blood pressure, and CVD risk outcomes.
2 型糖尿病患者常伴有合并的慢性疾病。我们开发了一种基于强化学习（RL）的人工智能算法，用于个性化的糖尿病和多种合并症管理，相对于当前的临床实践，具有显著改善健康结果的潜力。
我们将血糖、血压和心血管疾病（CVD）风险建模为健康结果，使用来自纽约大学朗格尼健康门诊电子病历的 2009-2017 年 16665 例 2 型糖尿病患者的回顾性队列。我们训练了一种 RL 处方算法，该算法使用患者在每次就诊时的个体特征和病史，推荐优化患者累积健康结果的治疗方案。RL 推荐在一个独立的患者子集上进行评估。
单结果优化 RL 算法，RL-血糖、RL-血压和 RL-CVD，分别在 86.1%、82.9%和 98.4%的就诊中推荐了与临床医生一致的处方。对于 RL 建议与临床医生处方不同的患者就诊，RL 算法下的就诊中血糖控制不佳（35%的就诊中 A1c>8%）、高血压控制不佳（16%的就诊中血压>140mmHg）和 CVD 风险高（25%的就诊中风险>20%）的就诊明显少于临床医生（分别为 43%、27%和 31%的就诊；均 p<0.001）。
用于 2 型糖尿病的个性化 RL 规定性框架与临床医生的处方具有高度一致性，并显著改善了血糖、血压和 CVD 风险结果。
