Inoue Tomohiro, Chen Yucan, Ohyanagi Toshio
Department of Psychology, The Chinese University of Hong Kong, Hong Kong SAR, China.
Department of Liberal Arts and Sciences, Sapporo Medical University, Sapporo, Japan.
Behav Res Methods. 2024 Dec 30;57(1):32. doi: 10.3758/s13428-024-02562-6.
Online language and literacy assessments have become prevalent in research and practice across settings. However, a notable exception is the assessment of handwriting and spelling, which has traditionally been conducted in person with paper and pencil. In light of this, we developed an automated, browser-based handwriting test application (Online Assessment of Handwriting and Spelling: OAHaS) for Japanese Kanji (Study 1) and examined its psychometric properties (Study 2). The automated scoring function using convolutional neural network (CNN) models achieved high recall (98.7%) and specificity (84.4%), as well as high agreement with manual scoring (95.4%). Additionally, behavioral validation with data from primary school children (N = 261, 49.0% female, age range = 6-12 years) indicated the high reliability and validity of our online test application, with a strong correlation between children's scores on the online and paper-based tests (r = .86). Moreover, our analysis indicated the potential utility of writing fluency measures (latency and duration) that are automatically recorded by OAHaS. Taken together, our browser-based application demonstrated the feasibility and viability of remote and automated assessment of handwriting skills, providing a streamlined approach to research and practice on handwriting. The source code of the application and supporting materials are available on Open Science Framework ( https://osf.io/gver2/ ).
在线语言和读写能力评估在各种研究和实践中已变得十分普遍。然而,一个显著的例外是手写和拼写评估,传统上这是通过纸笔进行的现场评估。有鉴于此,我们开发了一个基于浏览器的自动化手写测试应用程序(手写和拼写在线评估:OAHaS)用于日语汉字(研究1),并检验了其心理测量特性(研究2)。使用卷积神经网络(CNN)模型的自动评分功能实现了高召回率(98.7%)和高特异性(84.4%),以及与人工评分的高度一致性(95.4%)。此外,对小学生数据(N = 261,49.0%为女性,年龄范围 = 6至12岁)的行为验证表明,我们的在线测试应用程序具有高可靠性和有效性,儿童在在线测试和纸笔测试中的分数之间存在强相关性(r = 0.86)。此外,我们的分析表明了OAHaS自动记录的书写流畅性指标(潜伏期和持续时间)的潜在效用。综上所述,我们基于浏览器的应用程序证明了远程和自动化手写技能评估的可行性和实用性,为手写研究和实践提供了一种简化的方法。该应用程序的源代码和支持材料可在开放科学框架(https://osf.io/gver2/)上获取。