Department of Neuroscience, Research Group ExpORL, KU Leuven, Leuven, Belgium.
Cochlear Technology Centre, Schaliënhoevedreef 20i, Mechelen, Belgium.
Behav Res Methods. 2018 Oct;50(5):1921-1932. doi: 10.3758/s13428-017-0960-0.
The Language ENvironment Analysis system (LENA™) automatically analyzes the natural sound environments of children. Among other things, it estimates the amounts of adult words (AWC), child vocalizations (CV), conversational turns (CT), and electronic media (TV) that a child is exposed to. To assess LENA's reliability, we compared it to manual transcription. Specifically, we calculated the correlation and agreement between the LENA estimates and manual counts for 48 five-min audio samples. These samples were selected from eight day-long recordings of six Dutch-speaking children (ages 2-5). The correlations were strong for AWC, r = . 87, and CV, r = . 77, and comparatively low for CT, r = . 52, and TV, r = . 50. However, the agreement analysis revealed a constant bias in AWC counts, and proportional biases for CV and CT (i.e., the bias varied with the values for CV and CT). Agreement for detecting electronic media was poor. Moreover, the limits of agreement were wide for all four metrics. That is, the differences between LENA and the manual transcriptions for individual audio samples varied widely around the mean difference. This variation could indicate that LENA was affected by differences between the samples that did not equally affect the human transcribers. The disagreements and biases cast doubt on the comparability of LENA measurements across families and time, which is crucial for using LENA in research. Our sample is too small to conclude within which limits LENA's measurements are comparable, but it seems advisable to be cautious of factors that could systematically bias LENA's performance and thereby create confounds.
语言环境分析系统（LENA）自动分析儿童的自然语言环境。它可以估算儿童接触的成人词汇量（AWC）、儿童发声（CV）、对话轮次（CT）和电子媒体（电视）的数量。为了评估 LENA 的可靠性，我们将其与手动转录进行了比较。具体来说，我们计算了 48 个五分钟音频样本的 LENA 估计值与手动计数之间的相关性和一致性。这些样本是从六个讲荷兰语的儿童（年龄 2-5 岁）的八天全天录音中选择的。AWC 的相关性很强，r=0.87，CV 的相关性也很强，r=0.77，而 CT 的相关性相对较低，r=0.52，TV 的相关性也较低，r=0.50。然而，一致性分析显示 AWC 计数存在恒定的偏差，CV 和 CT 也存在比例偏差（即，偏差随 CV 和 CT 的值而变化）。检测电子媒体的一致性较差。此外，所有四个指标的差异范围都很宽。也就是说，LENA 与个别音频样本的手动转录之间的差异在平均值周围变化很大。这种变化可能表明 LENA 受到了样本之间差异的影响，而这些差异并没有平等地影响人工转录员。这些差异和偏差使人对 LENA 在家庭和时间上的测量结果的可比性产生了怀疑，这对于在研究中使用 LENA 至关重要。我们的样本太小，无法确定 LENA 测量结果的可比性范围，但谨慎考虑可能会系统地影响 LENA 性能并由此产生混淆的因素似乎是明智的。
