Tang Yaobin, Xu Yongze, Zhou Qunli, Bian Ran
Faculty of Psychology, Beijing Normal University, Beijing, China.
Department of Psychology, Faculty of Arts and Sciences, Beijing Normal University, Zhuhai, China.
JMIR Serious Games. 2025 Aug 25;13:e70453. doi: 10.2196/70453.
The traditional self-report instruments (eg, scales) used to measure antisocial personality traits are characterized by social desirability bias and fail to capture multidimensional behaviors (eg, manipulation and deception).
This study aimed to develop and validate an evidence-based design for a gamified assessment tool (Antisocial Personality Traits Evidence-Centered Design Gamified assessment tool; ASP-ECD-G) to measure 7 antisocial personality traits (manipulative, callous, deceptive, hostile, risk taking, impulsive, and irresponsible) as defined in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5).
This research featured a 3-phase evidence-centered design framework. Ontology development (study 1): semistructured interviews were conducted with 9 workplace professionals to translate the DSM-5 criteria into 24 observable workplace behaviors, which were integrated into a text-based game featuring 10 subscenarios, 34 interactive questions, and logic rooted in logical jumps to simulate real-world decision-making. Model construction (study 2): 6 machine learning models were trained by reference to a set of Personality Inventory for DSM-5 Short Form scores (n=286). The gated recurrent unit model, which uses 1-hot encoding to address nominal response data, was evaluated in terms of the root mean square error (RMSE), mean absolute error, criterion correlation (r), and test-retest reliability. Retest reliability was assessed using intraclass correlation coefficients based on 10 participants (1-month interval). Empirical validation (study 3): a 2×2 mixed design (n=148) was used to compare the gamified assessment with questionnaires under conditions involving incentives (ie, situations in which "rational results" led to increased payments).
For model performance, the gated recurrent unit outperformed the alternatives, as indicated by the highest criterion correlation (r=0.850) and the lowest test RMSE (0.273); in particular, it excelled in moderate score ranges (1.5-3, RMSE≤0.377) and in resisting extreme value distortions (3.5-4, RMSE 0.854). Retest reliability was moderate to strong (intraclass correlation coefficients=0.776, P=.02). For validation findings, the gamified assessment was associated with higher levels of immersion (mean 7.628 vs 7.216; F147=14.259, P<.001) and interest (mean 7.095 vs 6.155; F147=47.940, P<.001), although it also elicited stronger negative emotions (mean 4.365 vs 2.473; F147=151.109, P<.001). Incentives reduced questionnaire scores (incentivized: 2.066 vs control: 2.201; F1=5.740, P=.02) but had no effect on gamified scores (P=.71), confirming resistance to manipulation.
By integrating evidence-centered design with gamified workplace simulations, ASP-ECD-G can provide more objective and ecologically valid measurements of antisocial personality traits, thereby supporting both research and organizational practice.
Open Science Framework (OSF) Registries tvg6x; https://osf.io/tvg6x.
用于测量反社会人格特质的传统自我报告工具（如量表）存在社会赞许性偏差，且无法捕捉多维度行为（如操纵和欺骗）。
本研究旨在开发并验证一种基于证据的游戏化评估工具设计（反社会人格特质循证设计游戏化评估工具；ASP-ECD-G），以测量《精神疾病诊断与统计手册》第五版（DSM-5）中定义的7种反社会人格特质（操纵性、冷漠、欺骗性、敌意、冒险、冲动和不负责任）。
本研究采用了一个三阶段的循证设计框架。本体开发（研究1）：对9名职场专业人士进行半结构化访谈，将DSM-5标准转化为24种可观察到的职场行为，这些行为被整合到一个基于文本的游戏中，该游戏有10个子场景、34个互动问题，且逻辑基于逻辑跳转以模拟现实世界的决策。模型构建（研究2）：参照一组DSM-5简短形式人格量表得分（n = 286）训练6种机器学习模型。使用门控循环单元模型，该模型采用独热编码处理名义响应数据，并根据均方根误差（RMSE）、平均绝对误差、标准相关性（r）和重测信度进行评估。重测信度基于10名参与者（间隔1个月）使用组内相关系数进行评估。实证验证（研究3）：采用2×2混合设计（n = 148），在有激励措施（即“合理结果”会带来更多报酬的情况）的条件下，将游戏化评估与问卷进行比较。
就模型性能而言，门控循环单元模型优于其他模型，标准相关性最高（r = 0.850），测试RMSE最低（0.273）；特别是，它在中等分数范围（1.5 - 3，RMSE≤0.377）表现出色，且能抵抗极值扭曲（3.5 - 4，RMSE 0.854）。重测信度为中等至较强（组内相关系数 = 0.776，P = 0.02）。就验证结果而言，游戏化评估与更高水平的沉浸感（均值7.628对7.216；F147 = 14.259，P < 0.001）和兴趣（均值7.095对6.155；F147 = 47.940，P < 0.001）相关，尽管它也引发了更强烈的负面情绪（均值4.365对2.473；F147 = 151.109，P < 0.001）。激励措施降低了问卷得分（有激励：2.066对对照：2.201；F1 = 5.740，P = 0.02），但对游戏化得分没有影响（P = 0.71），证实了其对操纵的抗性。
通过将循证设计与游戏化职场模拟相结合，ASP-ECD-G能够更客观且生态效度更高地测量反社会人格特质，从而为研究和组织实践提供支持。
开放科学框架（OSF）注册库tvg6x；https://osf
.io/tvg6x
