Lin Sidian, Saghafian Soroush, Lipschitz Jessica M, Burdick Katherine E
The Kenneth C. Griffin Graduate School of Arts and Sciences, Harvard University, Cambridge, MA 02138, USA.
Harvard Kennedy School, Harvard University, Cambridge, MA 02138, USA.
PNAS Nexus. 2025 Aug 14;4(8):pgaf246. doi: 10.1093/pnasnexus/pgaf246. eCollection 2025 Aug.
This study introduces a novel multiagent reinforcement learning (MARL) algorithm designed for identifying and optimizing personalized recommendations in bipolar disorder. The algorithm leverages longitudinal offline data from wearables to recommend self-care strategies tailored to individual patients. We focus on self-care strategies involving physical activity (measured by steps), sleep duration, and bedtime consistency, aiming to reduce the periods of mood exacerbations. A key innovation of our MARL approach is the integration of copulas to model interagent dependencies, enhancing coordination among agents and improving policy learning. Findings suggest that following our algorithm's self-care recommendations could significantly reduce periods of elevated mood symptoms, resulting in improved overall well-being. Finally, the algorithm offers important clinical insights for treating bipolar patients, and shows promising theoretical properties independent of the specific application. Thus, this work not only advances MARL applications in personalized healthcare but also provides a new algorithmic approach for adaptive interventions in a wide range of chronic diseases.
本研究介绍了一种新颖的多智能体强化学习（MARL）算法，该算法旨在识别和优化双相情感障碍的个性化推荐。该算法利用可穿戴设备的纵向离线数据，为个体患者推荐量身定制的自我护理策略。我们专注于涉及身体活动（以步数衡量）、睡眠时间和就寝时间一致性的自我护理策略，旨在减少情绪恶化期。我们的MARL方法的一个关键创新是整合了copulas来建模智能体间的依赖性，增强智能体之间的协调并改善策略学习。研究结果表明，遵循我们算法的自我护理建议可以显著减少情绪症状升高期，从而改善整体幸福感。最后，该算法为治疗双相情感障碍患者提供了重要的临床见解，并显示出独立于特定应用的有前景的理论特性。因此，这项工作不仅推动了MARL在个性化医疗保健中的应用，还为广泛的慢性疾病的适应性干预提供了一种新的算法方法。
