Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA.
Department of Computer Science, Stanford University, Stanford, California, USA.
J Am Med Inform Assoc. 2020 Aug 1;27(8):1316-1320. doi: 10.1093/jamia/ocaa115.
Hand hygiene is essential for preventing hospital-acquired infections but is difficult to accurately track. The gold-standard (human auditors) is insufficient for assessing true overall compliance. Computer vision technology has the ability to perform more accurate appraisals. Our primary objective was to evaluate if a computer vision algorithm could accurately observe hand hygiene dispenser use in images captured by depth sensors.
Sixteen depth sensors were installed on one hospital unit. Images were collected continuously from March to August 2017. Utilizing a convolutional neural network, a machine learning algorithm was trained to detect hand hygiene dispenser use in the images. The algorithm's accuracy was then compared with simultaneous in-person observations of hand hygiene dispenser usage. Concordance rate between human observation and algorithm's assessment was calculated. Ground truth was established by blinded annotation of the entire image set. Sensitivity and specificity were calculated for both human and machine-level observation.
A concordance rate of 96.8% was observed between human and algorithm (kappa = 0.85). Concordance among the 3 independent auditors to establish ground truth was 95.4% (Fleiss's kappa = 0.87). Sensitivity and specificity of the machine learning algorithm were 92.1% and 98.3%, respectively. Human observations showed sensitivity and specificity of 85.2% and 99.4%, respectively.
A computer vision algorithm was equivalent to human observation in detecting hand hygiene dispenser use. Computer vision monitoring has the potential to provide a more complete appraisal of hand hygiene activity in hospitals than the current gold-standard given its ability for continuous coverage of a unit in space and time.
手部卫生对于预防医院获得性感染至关重要，但难以准确追踪。金标准（人工审核）不足以评估真实的总体依从性。计算机视觉技术具有进行更准确评估的能力。我们的主要目标是评估计算机视觉算法是否可以准确观察深度传感器拍摄的图像中的手部卫生消毒器使用情况。
在一个医院病房安装了 16 个深度传感器。2017 年 3 月至 8 月期间连续采集图像。利用卷积神经网络，训练机器学习算法以检测图像中手部卫生消毒器的使用情况。然后将算法的准确性与同时进行的手部卫生消毒器使用情况的人工观察进行比较。计算人类观察和算法评估之间的一致性率。通过对整个图像集进行盲法注释来确定真实情况。计算了人类和机器级观察的敏感性和特异性。
在人类和算法之间观察到 96.8%的一致性（kappa = 0.85）。建立真实情况的 3 位独立审核员之间的一致性为 95.4%（Fleiss's kappa = 0.87）。机器学习算法的敏感性和特异性分别为 92.1%和 98.3%。人工观察的敏感性和特异性分别为 85.2%和 99.4%。
计算机视觉算法在检测手部卫生消毒器使用方面与人工观察相当。与当前的金标准相比，计算机视觉监测具有在空间和时间上持续覆盖病房的能力，因此有可能提供对医院手部卫生活动更全面的评估。
