Department of Psychology, Virginia Commonwealth University, 806 West Franklin Street, PO Box 842018, Richmond, VA, 23284-2018, USA.
School of Education, Virginia Commonwealth University, Richmond, VA, USA.
Prev Sci. 2022 May;23(4):488-501. doi: 10.1007/s11121-021-01304-w. Epub 2021 Oct 29.
Though treatment integrity measurement is important for research intended to promote social and behavioral outcomes of children at risk for emotional and behavioral disorders (EBDs) in early childhood settings, measurement gaps exist in the field. This paper reports on the development and preliminary psychometric assessment of the treatment integrity measure for early childhood settings (TIMECS), an observational measure designed to address existing measurement gaps related to treatment integrity with tier 2 interventions in the early childhood field. To assess the preliminary score reliability (interrater) and validity (construct, discriminant) of the TIMECS, live observations (N = 650) in early childhood classrooms from 54 teachers (92.6% female, 7.4% male; 61.1% White) and 91 children (M age = 4.53 years, SD = .44; 45.1% female, 54.9% male; 45.1% Black) at risk for EBDs were scored by 12 coders using the TIMECS and an observational measure designed to assess teacher-child interactions. Teachers also self-reported on the quality of the teacher-child relationship. Interrater reliability (intraclass correlation coefficients, ICC [2,2]) for the quantity (i.e., adherence) item scores had a mean of .81 (SD = .07; range from .68 to .95), and the quality (i.e., competence) item scores had a mean of .69 (SD = .08; range from .52 to .80). Scores on the TIMECS Quantity and Quality items and scales showed evidence of construct validity, with the magnitude of the correlations suggesting that the quantity and quality items assess distinct components of treatment integrity. A TIMECS quantity scale also showed promise for intervention evaluation research by discriminating between teachers who had and had not been trained in a specific evidence-based intervention targeting social and behavioral skills in early childhood. The findings support the potential of the TIMECS to assess treatment integrity of teacher-delivered practices designed to address child social and behavioral outcomes of children at risk for EBDs in early childhood settings.
尽管治疗完整性测量对于旨在促进儿童早期环境中情绪和行为障碍（EBD）风险儿童的社会和行为结果的研究很重要，但该领域存在测量差距。本文报告了儿童早期环境治疗完整性测量工具（TIMECS）的开发和初步心理测量评估，这是一种观察性测量工具，旨在解决与儿童早期领域中二级干预措施的治疗完整性相关的现有测量差距。为了评估 TIMECS 的初步评分可靠性（评分者间）和有效性（结构，区分），对 54 名教师（92.6%为女性，7.4%为男性；61.1%为白人）和 91 名儿童（M 年龄=4.53 岁，SD=.44；45.1%为女性，54.9%为男性；45.1%为黑人）在早期儿童教室中的现场观察（N=650）进行了评分，这些儿童有 EBD 风险，由 12 名编码员使用 TIMECS 和旨在评估教师-儿童互动的观察性测量工具进行评分。教师还自我报告了教师-儿童关系的质量。数量（即，坚持）项目评分的评分者间可靠性（组内相关系数，ICC [2,2]）的平均值为.81（SD=.07；范围为.68 至.95），质量（即，能力）项目评分的平均值为.69（SD=.08；范围为.52 至.80）。TIMECS 数量和质量项目和量表的评分显示出结构有效性的证据，相关的大小表明数量和质量项目评估治疗完整性的不同组成部分。TIMECS 数量量表还通过区分接受和未接受特定基于证据的干预措施培训的教师，显示出干预评估研究的潜力，该干预措施针对儿童早期的社会和行为技能。研究结果支持 TIMECS 评估旨在解决儿童早期环境中 EBD 风险儿童社会和行为结果的教师实施实践的治疗完整性的潜力。
