Department of Computer Science, Stanford University, Stanford, CA, United States.
Department of Bioengineering, Stanford University, Stanford, CA, United States.
J Med Internet Res. 2022 Feb 15;24(2):e31830. doi: 10.2196/31830.
Autism spectrum disorder (ASD) is a widespread neurodevelopmental condition with a range of potential causes and symptoms. Standard diagnostic mechanisms for ASD, which involve lengthy parent questionnaires and clinical observation, often result in long waiting times for results. Recent advances in computer vision and mobile technology hold potential for speeding up the diagnostic process by enabling computational analysis of behavioral and social impairments from home videos. Such techniques can improve objectivity and contribute quantitatively to the diagnostic process.
In this work, we evaluate whether home videos collected from a game-based mobile app can be used to provide diagnostic insights into ASD. To the best of our knowledge, this is the first study attempting to identify potential social indicators of ASD from mobile phone videos without the use of eye-tracking hardware, manual annotations, and structured scenarios or clinical environments.
Here, we used a mobile health app to collect over 11 hours of video footage depicting 95 children engaged in gameplay in a natural home environment. We used automated data set annotations to analyze two social indicators that have previously been shown to differ between children with ASD and their neurotypical (NT) peers: (1) gaze fixation patterns, which represent regions of an individual's visual focus and (2) visual scanning methods, which refer to the ways in which individuals scan their surrounding environment. We compared the gaze fixation and visual scanning methods used by children during a 90-second gameplay video to identify statistically significant differences between the 2 cohorts; we then trained a long short-term memory (LSTM) neural network to determine if gaze indicators could be predictive of ASD.
Our results show that gaze fixation patterns differ between the 2 cohorts; specifically, we could identify 1 statistically significant region of fixation (P<.001). In addition, we also demonstrate that there are unique visual scanning patterns that exist for individuals with ASD when compared to NT children (P<.001). A deep learning model trained on coarse gaze fixation annotations demonstrates mild predictive power in identifying ASD.
Ultimately, our study demonstrates that heterogeneous video data sets collected from mobile devices hold potential for quantifying visual patterns and providing insights into ASD. We show the importance of automated labeling techniques in generating large-scale data sets while simultaneously preserving the privacy of participants, and we demonstrate that specific social engagement indicators associated with ASD can be identified and characterized using such data.
自闭症谱系障碍（ASD）是一种广泛存在的神经发育障碍，其潜在病因和症状多种多样。目前 ASD 的标准诊断机制包括家长填写冗长的问卷和临床观察，这通常会导致诊断结果的等待时间较长。计算机视觉和移动技术的最新进展为加速诊断过程提供了可能，这些技术可以通过分析家庭视频中的行为和社交障碍，实现计算分析。这些技术可以提高客观性，并为诊断过程提供定量贡献。
本研究旨在评估从基于游戏的移动应用程序中收集的家庭视频是否可用于提供 ASD 的诊断见解。据我们所知，这是第一项尝试从移动电话视频中识别潜在 ASD 社会指标的研究，而无需使用眼动追踪硬件、手动注释以及结构化场景或临床环境。
我们使用移动健康应用程序收集了超过 11 小时的视频片段，描绘了 95 名儿童在自然家庭环境中玩游戏的场景。我们使用自动化数据集注释来分析两个以前显示在 ASD 儿童和神经典型（NT）儿童之间存在差异的社会指标：（1）注视点模式，代表个体视觉焦点的区域；（2）视觉扫描方法，指个体扫描周围环境的方式。我们比较了儿童在 90 秒游戏视频中使用的注视点和视觉扫描方法，以确定这两个队列之间是否存在统计学上的显著差异；然后，我们训练一个长短期记忆（LSTM）神经网络，以确定注视指标是否可以预测 ASD。
我们的研究结果表明，两个队列之间的注视点模式存在差异；具体来说，我们可以确定 1 个具有统计学意义的注视区域（P<.001）。此外，我们还证明了 ASD 患者与 NT 儿童相比存在独特的视觉扫描模式（P<.001）。在基于粗略注视点注释训练的深度学习模型在识别 ASD 方面具有一定的预测能力。
最终，我们的研究表明，从移动设备收集的异构视频数据集具有量化视觉模式并提供 ASD 见解的潜力。我们展示了自动化标记技术在生成大规模数据集的同时保护参与者隐私的重要性，并且我们证明可以使用此类数据识别和描述与 ASD 相关的特定社交参与指标。
