Malorgio Amos, Henckert David, Schweiger Giovanna, Braun Julia, Zacharowski Kai, Raimann Florian J, Piekarski Florian, Meybohm Patrick, Hottenrott Sebastian, Froehlich Corinna, Spahn Donat R, Noethiger Christoph B, Tscholl David W, Roche Tadzio R
Institute of Anesthesiology, University Hospital Zurich, University of Zurich, 8091 Zurich, Switzerland.
Departments of Epidemiology and Biostatistics, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, 8001 Zurich, Switzerland.
Diagnostics (Basel). 2023 Oct 23;13(20):3281. doi: 10.3390/diagnostics13203281.
Machine learning can analyze vast amounts of data and make predictions for events in the future. Our group created machine learning models for vital sign predictions. To transport the information of these predictions without numbers and numerical values and make them easily usable for human caregivers, we aimed to integrate them into the Philips Visual-Patient-avatar, an avatar-based visualization of patient monitoring.
We conducted a computer-based simulation study with 70 participants in 3 European university hospitals. We validated the vital sign prediction visualizations by testing their identification by anesthesiologists and intensivists. Each prediction visualization consisted of a condition (e.g., low blood pressure) and an urgency (a visual indication of the timespan in which the condition is expected to occur). To obtain qualitative user feedback, we also conducted standardized interviews and derived statements that participants later rated in an online survey.
The mixed logistic regression model showed 77.9% (95% CI 73.2-82.0%) correct identification of prediction visualizations (i.e., condition and urgency both correctly identified) and 93.8% (95% CI 93.7-93.8%) for conditions only (i.e., without considering urgencies). A total of 49 out of 70 participants completed the online survey. The online survey participants agreed that the prediction visualizations were fun to use (32/49, 65.3%), and that they could imagine working with them in the future (30/49, 61.2%). They also agreed that identifying the urgencies was difficult (32/49, 65.3%).
This study found that care providers correctly identified >90% of the conditions (i.e., without considering urgencies). The accuracy of identification decreased when considering urgencies in addition to conditions. Therefore, in future development of the technology, we will focus on either only displaying conditions (without urgencies) or improving the visualizations of urgency to enhance usability for human users.
机器学习可以分析大量数据并对未来事件进行预测。我们团队创建了用于生命体征预测的机器学习模型。为了在不使用数字和数值的情况下传递这些预测信息，并使其便于护理人员使用，我们旨在将其集成到飞利浦视觉患者化身中，这是一种基于化身的患者监测可视化工具。
我们在3家欧洲大学医院对70名参与者进行了基于计算机的模拟研究。我们通过测试麻醉师和重症监护医生对生命体征预测可视化的识别来验证它们。每个预测可视化包括一种状况（例如，低血压）和一种紧急程度（对预期出现该状况的时间跨度的视觉指示）。为了获得定性的用户反馈，我们还进行了标准化访谈并得出陈述，参与者随后在在线调查中对这些陈述进行评分。
混合逻辑回归模型显示，预测可视化（即状况和紧急程度均正确识别）的正确识别率为77.9%（95%置信区间73.2 - 82.0%），仅状况（即不考虑紧急程度）的正确识别率为93.8%（95%置信区间93.7 - 93.8%）。70名参与者中有49人完成了在线调查。在线调查参与者一致认为预测可视化使用起来很有趣（32/49，65.3%），并且他们可以想象未来使用它们（30/49，61.2%）。他们还一致认为识别紧急程度很困难（32/49，65.3%）。
本研究发现护理人员正确识别了超过90%的状况（即不考虑紧急程度）。当除了状况之外还考虑紧急程度时，识别准确率会降低。因此，在该技术的未来发展中，我们将专注于要么仅显示状况（不显示紧急程度），要么改进紧急程度的可视化以提高对人类用户的可用性。
