Wawer Aleksander, Chojnicka Izabela, Sarzyńska-Wawer Justyna, Krawczyk Małgorzata
Polish Academy of Sciences, Institute of Computer Science, Warsaw, Poland.
Department of Health and Rehabilitation Psychology, Faculty of Psychology, University of Warsaw, Warsaw, Poland.
Acta Psychiatr Scand. 2025 Mar;151(3):259-269. doi: 10.1111/acps.13737. Epub 2024 Jul 20.
The goals of this article are as follows. First, to investigate the possibility of detecting autism spectrum disorder (ASD) from text data using the latest generation of machine learning tools. Second, to compare model performance on two datasets of transcribed statements, collected using two different diagnostic tools. Third, to investigate the feasibility of knowledge transfer between models trained on both datasets and check if data augmentation can help alleviate the problem of a small number of observations.
We explore two techniques to detect ASD. The first one is based on fine-tuning HerBERT, a BERT-based, monolingual deep transformer neural network. The second one uses the newest, multipurpose text embeddings from OpenAI and a classifier. We apply the methods to two separate datasets of transcribed statements, collected using two different diagnostic tools: thought, language, and communication (TLC) and autism diagnosis observation schedule-2 (ADOS-2). We conducted several cross-dataset experiments in both a zero-shot setting and a setting where models are pretrained on one dataset and then training continues on another to test the possibility of knowledge transfer.
Unlike previous studies, the models we tested obtained average results on ADOS-2 data but reached very good performance of the models in TLC. We did not observe any benefits from knowledge transfer between datasets. We observed relatively poor performance of models trained on augmented data and hypothesize that data augmentation by back translation obfuscates autism-specific signals.
The quality of machine learning models that detect ASD from text data is improving, but model results are dependent on the type of input data or diagnostic tool.
本文的目标如下。其一，使用最新一代机器学习工具研究从文本数据中检测自闭症谱系障碍（ASD）的可能性。其二，比较在使用两种不同诊断工具收集的两个转录陈述数据集上的模型性能。其三，研究在两个数据集上训练的模型之间知识转移的可行性，并检查数据增强是否有助于缓解观测数据量少的问题。
我们探索了两种检测ASD的技术。第一种基于对HerBERT（一种基于BERT的单语深度变换器神经网络）进行微调。第二种使用来自OpenAI的最新多用途文本嵌入和一个分类器。我们将这些方法应用于使用两种不同诊断工具收集的两个单独的转录陈述数据集：思维、语言和沟通（TLC）以及自闭症诊断观察量表-2（ADOS-2）。我们在零样本设置以及模型在一个数据集上进行预训练然后在另一个数据集上继续训练的设置下进行了几次跨数据集实验，以测试知识转移的可能性。
与先前的研究不同，我们测试的模型在ADOS-2数据上获得了平均结果，但在TLC数据上模型达到了非常好的性能。我们没有观察到数据集之间知识转移带来的任何益处。我们观察到在增强数据上训练的模型性能相对较差，并推测通过反向翻译进行数据增强会模糊自闭症特异性信号。
从文本数据中检测ASD的机器学习模型的质量正在提高，但模型结果取决于输入数据的类型或诊断工具。
