Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Hong Kong, China.
Department of Anaesthesia and Intensive Care, The Chinese University of Hong Kong, Hong Kong, China.
PLoS One. 2021 May 13;16(5):e0251550. doi: 10.1371/journal.pone.0251550. eCollection 2021.
Unprecedented public health measures have been used during this coronavirus 2019 (COVID-19) pandemic to control the spread of SARS-CoV-2 virus. It is a challenge to implement timely and appropriate public health interventions.
Population and COVID-19 epidemiological data between 21st January 2020 to 15th November 2020 from 216 countries and territories were included with the implemented public health interventions. We used deep reinforcement learning, and the algorithm was trained to enable agents to try to find optimal public health strategies that maximized total reward on controlling the spread of COVID-19. The results suggested by the algorithm were analyzed against the actual timing and intensity of lockdown and travel restrictions. Early implementations of the actual lockdown and travel restriction policies, usually at the time of local index case were associated with less burden of COVID-19. In contrast, our agent suggested to initiate at least minimal intensity of lockdown or travel restriction even before or on the day of the index case in each country and territory. In addition, the agent mostly recommended a combination of lockdown and travel restrictions and higher intensity policies than the policies implemented by governments, but did not always encourage rapid full lockdown and full border closures. The limitation of this study was that it was done with incomplete data due to the emerging COVID-19 epidemic, inconsistent testing and reporting. In addition, our research focuses only on population health benefits by controlling the spread of COVID-19 without balancing the negative impacts of economic and social consequences.
Compared to actual government implementation, our algorithm mostly recommended earlier intensity of lockdown and travel restrictions. Reinforcement learning may be used as a decision support tool for implementation of public health interventions during COVID-19 and future pandemics.
在这次 2019 年冠状病毒病（COVID-19）大流行期间，前所未有地采取了公共卫生措施来控制 SARS-CoV-2 病毒的传播。实施及时和适当的公共卫生干预措施是一项挑战。
纳入了 2020 年 1 月 21 日至 2020 年 11 月 15 日来自 216 个国家和地区的人群和 COVID-19 流行病学数据，以及实施的公共卫生干预措施。我们使用深度强化学习，让算法尝试寻找最佳公共卫生策略，使控制 COVID-19 传播的总回报最大化。对算法建议的结果进行了分析，以评估其与实际封锁和旅行限制的时间和强度。尽早实施实际的封锁和旅行限制政策，通常在当地首例病例发生时，与 COVID-19 的负担较小有关。相比之下，我们的代理建议在每个国家和地区，即使在首例病例之前或当天，也至少要实施最低限度的封锁或旅行限制。此外，代理通常建议实施封锁和旅行限制相结合，以及比政府实施的政策更高强度的政策，但并不总是鼓励快速全面封锁和全面关闭边境。本研究的局限性在于，由于 COVID-19 疫情的出现、检测和报告不一致，数据不完整。此外，我们的研究仅关注通过控制 COVID-19 传播来控制人口健康的益处，而没有平衡经济和社会后果的负面影响。
与实际政府实施相比，我们的算法主要建议更早地实施封锁和旅行限制。强化学习可作为在 COVID-19 和未来大流行期间实施公共卫生干预措施的决策支持工具。
