Bilinski Alyssa M, Salomon Joshua A, Hatfield Laura A
Departments of Health Services, Policy and Practice & Biostatistics, Brown University, 121 S. Main St., Providence, RI 02912 USA.
Department of Health Policy, Stanford University, Stanford, CA 94305 USA.
medRxiv. 2023 Feb 16:2023.02.15.23285969. doi: 10.1101/2023.02.15.23285969.
Throughout the COVID-19 pandemic, policymakers have proposed risk metrics, such as the CDC Community Levels, to guide local and state decision-making. However, risk metrics have not reliably predicted key outcomes and often lack transparency in terms of prioritization of false positive versus false negative signals. They have also struggled to maintain relevance over time due to slow and infrequent updates addressing new variants and shifts in vaccine- and infection-induced immunity. We make two contributions to address these weaknesses of risk metrics. We first present a framework to evaluate predictive accuracy based on policy targets related to severe disease and mortality, allowing for explicit preferences toward false negative versus false positive signals. This approach allows policymakers to optimize metrics for specific preferences and interventions. Second, we propose a novel method to update risk thresholds in real-time. We show that this adaptive approach to designating areas as "high risk" improves performance over static metrics in predicting 3-week-ahead mortality and intensive care usage at both state and county levels. We also demonstrate that with our approach, using only new hospital admissions to predict 3-week-ahead mortality and intensive care usage has performed consistently as well as metrics that also include cases and inpatient bed usage. Our results highlight that a key challenge for COVID-19 risk prediction is the changing relationship between indicators and outcomes of policy interest. Adaptive metrics therefore have a unique advantage in a rapidly evolving pandemic context.
In the rapidly-evolving COVID-19 pandemic, public health risk metrics often become less relevant over time. Risk metrics are designed to predict future severe disease and mortality based on currently-available surveillance data, such as cases and hospitalizations. However, the relationship between cases, hospitalizations, and mortality has varied considerably over the course of the pandemic, in the context of new variants and shifts in vaccine- and infection-induced immunity. We propose an adaptive approach that regularly updates metrics based on the relationship between surveillance inputs and future outcomes of policy interest. Our method captures changing pandemic dynamics, requires only hospitalization input data, and outperforms static risk metrics in predicting high-risk states and counties.
在整个新冠疫情期间，政策制定者提出了风险指标，如美国疾病控制与预防中心（CDC）的社区风险等级，以指导地方和州的决策。然而，风险指标并未可靠地预测关键结果，而且在假阳性信号与假阴性信号的优先级方面往往缺乏透明度。由于针对新变种以及疫苗和感染诱导免疫的变化进行更新的速度缓慢且不频繁，它们还难以长期保持相关性。我们针对风险指标的这些弱点做出了两项贡献。我们首先提出了一个基于与严重疾病和死亡率相关的政策目标来评估预测准确性的框架，允许对假阴性信号与假阳性信号有明确的偏好。这种方法使政策制定者能够针对特定偏好和干预措施优化指标。其次，我们提出了一种实时更新风险阈值的新方法。我们表明，这种将地区指定为“高风险”的自适应方法在预测州和县层面提前3周的死亡率和重症监护使用率方面比静态指标表现更好。我们还证明，采用我们的方法，仅使用新入院人数来预测提前3周的死亡率和重症监护使用率，其表现与同时包含病例数和住院床位使用率的指标一样稳定。我们的结果凸显出，新冠风险预测的一个关键挑战在于政策关注的指标与结果之间不断变化的关系。因此，在疫情迅速演变的背景下，自适应指标具有独特优势。
在迅速演变的新冠疫情中，公共卫生风险指标往往会随着时间推移而变得相关性降低。风险指标旨在根据当前可用的监测数据（如病例数和住院人数）预测未来的严重疾病和死亡率。然而，在新变种以及疫苗和感染诱导免疫发生变化的背景下，病例数、住院人数和死亡率之间的关系在疫情过程中发生了很大变化。我们提出一种自适应方法，该方法根据监测输入与政策关注的未来结果之间的关系定期更新指标。我们的方法捕捉了不断变化的疫情动态，仅需要住院输入数据，并且在预测高风险州和县方面优于静态风险指标。
