Hurvitz Brain Sciences Research Program, Sunnybrook Research Institute, Toronto, Ontario, Canada.
Department of Speech-Language Pathology, Rehabilitation Sciences Institute, University of Toronto, Ontario, Canada.
Am J Speech Lang Pathol. 2023 Aug 17;32(4S):1884-1900. doi: 10.1044/2023_AJSLP-22-00177. Epub 2023 Jul 26.
The primary aim of this study was to establish the reliability of candidate items as a step in the development of the Amyotrophic Lateral Sclerosis-Bulbar Dysfunction Index-Remote (ALS-BDI-Remote), a novel tool being developed for the detection and monitoring of bulbar signs and symptoms in remote settings.
The set of candidate items included 40 items covering three domains: cranial nerve examination, auditory-perceptual evaluation, and functional assessment. Forty-eight participants diagnosed with ALS and exhibiting a range of bulbar disease severity were included. Data collection for each participant took place on Zoom over three sessions. During Session 1, the participants were instructed to adjust their Zoom settings and to optimize their recording environment (e.g., lighting, background noise). Their cognition and eating were screened to determine their ability to follow instructions and their eligibility to perform the swallowing and chewing tasks. During Session 2, two speech-language pathologists (SLPs) administered the tool consecutively to determine the items' interrater reliability. During Session 3, one of the SLPs readministered the tool within 2 weeks of Session 1 to assess test-retest reliability. The reliability of each item was estimated using weighted kappa and the percentage of agreement. To be considered reliable, the items had to reach a threshold of 0.5 weighted kappa or 80% percentage agreement (if skewed distribution of the scores) for both interrater and test-retest reliability.
In total, 33 of the 40 candidate items reached the reliability cutoff for both reliability analyses. All assessment domains included reliable items. Items requiring very good visualization of structures or movements were generally less reliable.
This study resulted in the selection of reliable items to be included in the next version of the ALS-BDI-Remote, which will undergo psychometric evaluation (reliability, validity, and responsiveness analyses). Additionally, the results contributed to our understanding of the remote administration of SLP assessments for telehealth applications.
本研究的主要目的是确定候选项目的可靠性，这是开发肌萎缩侧索硬化症-延髓功能障碍远程评估量表（ALS-BDI-Remote）的一个步骤，该量表是一种新工具，旨在远程环境中检测和监测延髓体征和症状。
候选项目集包括 40 个项目，涵盖三个领域：颅神经检查、听觉感知评估和功能评估。共纳入 48 名诊断为 ALS 且延髓疾病严重程度不同的参与者。每位参与者的数据采集分三个会话在 Zoom 上进行。在第 1 会话中，参与者被指示调整他们的 Zoom 设置并优化他们的录制环境（例如，照明、背景噪音）。他们的认知和进食能力进行筛查，以确定他们是否能够按照说明进行操作，以及他们是否有能力进行吞咽和咀嚼任务。在第 2 会话中，两名言语语言病理学家（SLP）连续使用该工具，以确定项目的组内可靠性。在第 3 会话中，其中一位 SLP 在第 1 会话后的 2 周内重新使用该工具，以评估测试-重测可靠性。使用加权 Kappa 和一致性百分比来估计每个项目的可靠性。为了被认为是可靠的，项目必须在组内可靠性和测试-重测可靠性分析中达到加权 Kappa 为 0.5 或一致性百分比为 80%的阈值（如果得分呈偏态分布）。
总共 40 个候选项目中有 33 个项目在两项可靠性分析中都达到了可靠性标准。所有评估领域都包括可靠的项目。需要非常好地可视化结构或运动的项目通常不太可靠。
本研究选择了可靠的项目纳入下一版 ALS-BDI-Remote，该量表将进行心理测量学评估（可靠性、有效性和反应性分析）。此外，研究结果有助于我们理解远程言语语言病理评估在远程医疗应用中的应用。
