Gale Robert, Bird Julie, Wang Yiyi, van Santen Jan, Prud'hommeaux Emily, Dolata Jill, Asgari Meysam
Center for Spoken Language Understanding, Oregon Health & Science University (OHSU), Portland, OR, United States.
Boston College, Chestnut Hill, MA, United States.
Front Psychol. 2021 Jul 22;12:668401. doi: 10.3389/fpsyg.2021.668401. eCollection 2021.
Speech and language impairments are common pediatric conditions, with as many as 10% of children experiencing one or both at some point during development. Expressive language disorders in particular often go undiagnosed, underscoring the immediate need for assessments of expressive language that can be administered and scored reliably and objectively. In this paper, we present a set of highly accurate computational models for automatically scoring several common expressive language tasks. In our assessment framework, instructions and stimuli are presented to the child on a tablet computer, which records the child's responses in real time, while a clinician controls the pace and presentation of the tasks using a second tablet. The recorded responses for four distinct expressive language tasks (expressive vocabulary, word structure, recalling sentences, and formulated sentences) are then scored using traditional paper-and-pencil scoring and using machine learning methods relying on a deep neural network-based language representation model. All four tasks can be scored automatically from both clean and verbatim speech transcripts with very high accuracy at the item level (83-99%). In addition, these automated scores correlate strongly and significantly (ρ = 0.76-0.99, < 0.001) with manual item-level, raw, and scaled scores. These results point to the utility and potential of automated computationally-driven methods of both administering and scoring expressive language tasks for pediatric developmental language evaluation.
言语和语言障碍是常见的儿科疾病，多达10%的儿童在发育过程中的某个阶段会出现其中一种或两种障碍。特别是表达性语言障碍常常未被诊断出来，这凸显了对能够可靠、客观地进行管理和评分的表达性语言评估的迫切需求。在本文中，我们提出了一组高度准确的计算模型，用于自动对几种常见的表达性语言任务进行评分。在我们的评估框架中，通过平板电脑向儿童呈现指令和刺激，平板电脑实时记录儿童的反应，而临床医生则使用另一台平板电脑控制任务的节奏和呈现方式。然后，使用传统的纸笔评分方法以及依赖基于深度神经网络的语言表示模型的机器学习方法，对四个不同的表达性语言任务（表达性词汇、单词结构、回忆句子和造句）的记录反应进行评分。所有这四个任务都可以在项目层面以非常高的准确率（83 - 99%）从清晰语音和逐字语音转录本中自动评分。此外，这些自动评分与人工项目层面的原始分数和标准化分数高度相关且具有显著性（ρ = 0.76 - 0.99，< 0.001）。这些结果表明，在儿科发育语言评估中，由计算驱动的自动管理和评分表达性语言任务的方法具有实用性和潜力。
