Basha Sydni A J, Cai Qiyue, Lee Susanne, Tran Tiffany, Majerle Amy, Tiede Shauna, Gewirtz Abigail H
The REACH Institute, Department of Psychology, Arizona State University, 900 S. McAllister Avenue, Suite 205, Tempe, AZ, 85281, USA.
Paul Baerwald School of Social Work and Social Welfare, Hebrew University of Jerusalem, Jerusalem, Israel.
Prev Sci. 2024 Jul 12. doi: 10.1007/s11121-024-01706-6.
Many conventional research methods employed in randomized controlled trials were not possible during the height of the COVID-19 pandemic. In particular, behavioral observations are nearly universally gathered in-person. Observational methods are valued for the rich, informative data they produce in comparison to non-observational methods and are a cornerstone of parenting and family research. COVID provided the opportunity to, and indeed necessitated, the transition to fully remote observation. However, little to no studies have investigated whether remotely collected observational data are methodologically sound. This paper assesses the feasibility of remote data collection by describing the transition between in-person and fully remote observational data collection during a Sequential, Multiple Assignment, Randomized Trial (SMART) of a parenting program that took place both before and during the pandemic. Using mixed-methods data from coders, the overall quality of video-recorded data collected both before and during COVID was examined. Coder reliability over time was assessed with intraclass correlation coefficients. Results suggest that the frequency of audio problems, the severity of visual problems, and the level of administration challenges decreased after transitioning to remote data collection. Additionally, coders showed good to excellent reliability coding remotely collected data, and reliability even improved on some measured tasks. Although challenges to remote data collection exist, this study demonstrated that observational data can be collected feasibly and reliably. As observational data collection is a key method to assess parenting practices, these findings should improve researcher confidence in utilizing remote observational methods in prevention science.
在新冠疫情高峰期，随机对照试验中使用的许多传统研究方法都无法实施。特别是，行为观察几乎都是通过面对面方式进行的。与非观察性方法相比，观察性方法因其能产生丰富、信息量大的数据而受到重视，是育儿和家庭研究的基石。新冠疫情为向完全远程观察的转变提供了契机，而且实际上使之成为必要。然而，几乎没有研究调查过远程收集的观察数据在方法上是否合理。本文通过描述在疫情之前和期间进行的一项育儿项目的序贯、多重分配、随机试验（SMART）中，从面对面观察数据收集到完全远程观察数据收集的转变，评估了远程数据收集的可行性。利用编码员的混合方法数据，对疫情前和疫情期间收集的视频记录数据的整体质量进行了检查。通过组内相关系数评估编码员随时间的可靠性。结果表明，在过渡到远程数据收集后，音频问题的频率、视觉问题的严重程度以及管理挑战的水平都有所下降。此外，编码员对远程收集的数据进行编码时表现出良好到优秀的可靠性，甚至在一些测量任务上可靠性有所提高。尽管远程数据收集存在挑战，但本研究表明观察数据可以可行且可靠地收集。由于观察数据收集是评估育儿实践的关键方法，这些发现应能增强研究人员在预防科学中使用远程观察方法的信心。
