IEEE Trans Biomed Eng. 2021 Feb;68(2):664-672. doi: 10.1109/TBME.2020.3010472. Epub 2021 Jan 21.
Major depressive disorder (MDD) is a common psychiatric disorder that leads to persistent changes in mood and interest among other signs and symptoms. We hypothesized that convolutional neural network (CNN) based automated facial expression recognition, pre-trained on an enormous auxiliary public dataset, could provide improve generalizable approach to MDD automatic assessment from videos, and classify remission or response to treatment.
We evaluated a novel deep neural network framework on 365 video interviews (88 hours) from a cohort of 12 depressed patients before and after deep brain stimulation (DBS) treatment. Seven basic emotions were extracted with a Regional CNN detector and an Imagenet pre-trained CNN, both of which were trained on large-scale public datasets (comprising over a million images). Facial action units were also extracted with the Openface toolbox. Statistics of the temporal evolution of these image features over each recording were extracted and used to classify MDD remission and response to DBS treatment.
An Area Under the Curve of 0.72 was achieved using leave-one-subject-out cross-validation for remission classification and 0.75 for response to treatment.
This work demonstrates the potential for the classification of MDD remission and response to DBS treatment from passively acquired video captured during unstructured, unscripted psychiatric interviews.
This novel MDD evaluation could be used to augment current psychiatric evaluations and allow automatic, low-cost, frequent use when an expert isn't readily available or the patient is unwilling or unable to engage. Potentially, the framework may also be applied to other psychiatric disorders.
重度抑郁症（MDD）是一种常见的精神障碍，会导致情绪和兴趣的持续变化以及其他迹象和症状。我们假设基于卷积神经网络（CNN）的自动面部表情识别，在庞大的辅助公共数据集上进行预训练，可以提供改善从视频中进行 MDD 自动评估的可推广方法，并对治疗的缓解或反应进行分类。
我们在接受深部脑刺激（DBS）治疗前后，对 12 名抑郁患者的 365 个视频访谈（88 小时）评估了一种新的深度神经网络框架。通过区域 CNN 检测器和 Imagenet 预训练 CNN 提取七种基本情绪，这两种检测器都在大规模公共数据集（包含超过一百万张图像）上进行训练。还使用 Openface 工具箱提取面部动作单元。提取每个记录中这些图像特征的时间演变的统计信息，并用于分类 MDD 的缓解和对 DBS 治疗的反应。
使用一次剔除一位受试者的交叉验证，对缓解的分类获得了 0.72 的曲线下面积，对治疗反应的分类获得了 0.75 的曲线下面积。
这项工作表明，从非结构化、非脚本化的精神病访谈中被动获取的视频中，对 MDD 缓解和对 DBS 治疗的反应进行分类是有可能的。这种新颖的 MDD 评估方法可以用于补充当前的精神病评估，并且可以在没有专家的情况下，或患者不愿意或无法参与时，自动、低成本、频繁使用。此外，该框架还可能适用于其他精神障碍。
