Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, New South Wales, Australia.
J Am Med Inform Assoc. 2022 Jul 12;29(8):1400-1408. doi: 10.1093/jamia/ocac071.
People are increasingly encouraged to self-manage their chronic conditions; however, many struggle to practise it effectively. Most studies that investigate patient work (ie, tasks involved in self-management and contexts influencing such tasks) rely on self-reports, which are subject to recall and other biases. Few studies use wearable cameras and deep learning to capture and classify patient work activities automatically.
We propose a deep learning approach to classify activities of patient work collected from wearable cameras, thereby studying self-management routines more effectively. Twenty-six people with type 2 diabetes and comorbidities wore a wearable camera for a day, generating more than 400 h of video across 12 daily activities. To classify these video images, a weighted ensemble network that combines Linear Discriminant Analysis, Deep Convolutional Neural Networks, and Object Detection algorithms is developed. Performance of our model is assessed using Top-1 and Top-5 metrics, compared against manual classification conducted by 2 independent researchers.
Across 12 daily activities, our model achieved on average the best Top-1 and Top-5 scores of 81.9 and 86.8, respectively. Our model also outperformed other non-ensemble techniques in terms of Top-1 and Top-5 scores for most activity classes, demonstrating the superiority of leveraging weighted ensemble techniques.
Deep learning can be used to automatically classify daily activities of patient work collected from wearable cameras with high levels of accuracy. Using wearable cameras and a deep learning approach can offer an alternative approach to investigate patient work, one not subjected to biases commonly associated with self-report methods.
人们越来越多地被鼓励自我管理慢性疾病；然而，许多人难以有效地进行实践。大多数研究患者工作（即自我管理所涉及的任务和影响这些任务的背景）都依赖于自我报告，这容易受到回忆和其他偏差的影响。很少有研究使用可穿戴相机和深度学习来自动捕捉和分类患者工作活动。
我们提出了一种深度学习方法来对可穿戴相机采集的患者工作活动进行分类，从而更有效地研究自我管理常规。26 名 2 型糖尿病合并症患者佩戴了可穿戴相机一天，生成了超过 400 小时的 12 项日常活动视频。为了对这些视频图像进行分类，开发了一个结合线性判别分析、深度卷积神经网络和目标检测算法的加权集成网络。我们使用 Top-1 和 Top-5 指标评估模型的性能，并与由 2 位独立研究人员进行的手动分类进行比较。
在 12 项日常活动中，我们的模型平均达到了最佳的 Top-1 和 Top-5 分数，分别为 81.9 和 86.8。在大多数活动类别中，我们的模型在 Top-1 和 Top-5 分数方面也优于其他非集成技术，这表明利用加权集成技术具有优越性。
深度学习可用于从可穿戴相机自动分类患者工作的日常活动，具有很高的准确性。使用可穿戴相机和深度学习方法可以提供一种替代方法来研究患者工作，这种方法不会受到自我报告方法常见的偏差的影响。
