Center on the Ecology of Early Development, Boston University, MA.
Department of Mechanical Engineering, University of Michigan, Ann Arbor.
J Speech Lang Hear Res. 2024 Aug 5;67(8):2669-2684. doi: 10.1044/2024_JSLHR-23-00310. Epub 2024 Jul 17.
This study examines the accuracy of Interaction Detection in Early Childhood Settings (IDEAS), a program that automatically transcribes audio files and estimates linguistic units relevant to speech-language therapy, including part-of-speech units that represent features of language complexity, such as adjectives and coordinating conjunctions.
Forty-five video-recorded speech-language therapy sessions involving 27 speech-language pathologists (SLPs) and 56 children were used. The measure determines the accuracy of IDEAS diarization (i.e., speech segmentation and speaker classification). Two additional evaluation metrics, namely, median absolute relative error and correlation, indicate the accuracy of IDEAS for the estimation of linguistic units as compared with two conditions, namely, Oracle (manual diarization) and Voice Type Classifier (existing diarizer with acceptable accuracy).
The high measure for SLP talk data suggests high accuracy of IDEAS diarization for SLP talk but less so for child talk. These differences are reflected in the accuracy of IDEAS linguistic unit estimates. IDEAS median absolute relative error and correlation values for nine of the 10 SLP linguistic unit estimates meet the accuracy criteria, but none of the child linguistic unit estimates meet these criteria. The type of linguistic units also affects IDEAS accuracy.
IDEAS was tailored to educational settings to automatically convert audio recordings into text and to provide linguistic unit estimates in speech-language therapy sessions and classroom settings. Although not perfect, IDEAS is reliable in automatically capturing and returning linguistic units, especially in SLP talk, that are relevant in research and practice. The tool offers a way to automatically measure SLP talk in clinical settings, which will support research seeking to understand how SLP talk influences children's language growth.
本研究考察了 Interaction Detection in Early Childhood Settings（IDEAS）的准确性，该程序可以自动转录音频文件，并估计与言语治疗相关的语言单位，包括代表语言复杂度特征的词性单位，如形容词和并列连词。
使用了 45 节视频记录的言语治疗会话，涉及 27 名言语治疗师（SLP）和 56 名儿童。该指标用于确定 IDEAS 话轮检测（即语音分割和说话人分类）的准确性。另外两个评估指标，即中位数绝对相对误差和相关性，表明与两种情况相比，即 Oracle（手动话轮检测）和 Voice Type Classifier（具有可接受准确性的现有话轮检测），IDEAS 用于语言单位估计的准确性。
对于 SLP 谈话数据，高的指标表明 IDEAS 对 SLP 谈话的话轮检测具有高度准确性，但对儿童谈话的准确性则较低。这些差异反映在 IDEAS 语言单位估计的准确性上。IDEAS 对 10 个 SLP 语言单位估计中的 9 个的中位数绝对相对误差和相关性值符合准确性标准，但没有一个儿童语言单位估计符合这些标准。语言单位的类型也会影响 IDEAS 的准确性。
IDEAS 是为教育环境量身定制的，可自动将录音转换为文本，并在言语治疗会话和课堂环境中提供语言单位估计。尽管不是完美的，但 IDEAS 在自动捕获和返回相关语言单位方面是可靠的，特别是在 SLP 谈话中，这在研究和实践中都很重要。该工具提供了一种在临床环境中自动测量 SLP 谈话的方法，这将支持研究，以了解 SLP 谈话如何影响儿童的语言发展。
