Zhang Yanyi, Gu Yue, Marsic Ivan, Zheng Yinan, Burd Randall S
Department of Electrical and Computer Engineering, Rutgers University, Piscataway, NJ, USA.
Division of Trauma and Burn Surgery, Children's National Medical Center, Washington, DC, USA.
Proc (IEEE Int Conf Healthc Inform). 2020 Nov-Dec;2020. doi: 10.1109/ichi48887.2020.9374399. Epub 2021 Mar 12.
We introduce a video-based system for concurrent activity recognition during teamwork in a clinical setting. During system development, we preserved patient and provider privacy by pre-computing spatio-temporal features. We extended the inflated 3D ConvNet (i3D) model for concurrent activity recognition. For the model training, we tuned the weights of the final stages of i3D using back-propagated loss from the fully-connected layer. We applied filtering on the model predictions to remove noisy predictions. We evaluated the system on five activities performed during trauma resuscitation, the initial management of injured patients in the emergency department. Our system achieved an average value of 74% average precision (AP) for these five activities and outperformed previous systems designed for the same domain. We visualized feature maps from the model, showing that the system learned to focus on regions relevant to performance of each activity.
我们介绍了一种基于视频的系统，用于在临床环境中的团队协作期间进行并发活动识别。在系统开发过程中，我们通过预计算时空特征来保护患者和医护人员的隐私。我们扩展了用于并发活动识别的膨胀3D卷积神经网络（i3D）模型。对于模型训练，我们使用来自全连接层的反向传播损失来调整i3D最后阶段的权重。我们对模型预测应用了滤波以去除噪声预测。我们在创伤复苏（急诊科对受伤患者的初始处理）期间执行的五项活动上对该系统进行了评估。我们的系统在这五项活动上实现了平均精度（AP）为74%的平均值，并且优于为同一领域设计的先前系统。我们可视化了模型的特征图，表明该系统学会了专注于与每项活动执行相关的区域。
