Langezaal Mathijs A, van den Broek Egon L, Rey Grégoire, Le Moual Nicole, Pilorget Corinne, Goldberg Marcel, Vermeulen Roel, Peters Susan
Population-Based Epidemiological Cohorts Unit UMS11, INSERM, Villejuif, France
Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands.
Occup Environ Med. 2025 Jul 9;82(4):183-190. doi: 10.1136/oemed-2024-109823.
The manual coding of job descriptions is time-consuming, expensive and requires expert knowledge. Decision support systems (DSS) provide a valuable alternative by offering automated suggestions that support decision-making, improving efficiency while allowing manual corrections to ensure reliability. However, this claim has not been proven with expert coders. This study aims to fill this omission by comparing manual with decision-supported coding, using the new DSS OPERAS.
Five expert coders proficient in using the French classification systems for occupations PCS2003 and activity sectors NAF2008 each successively coded two subsets of job descriptions from the CONSTANCES cohort manually and using OPERAS. Subsequently, we assessed coding time and inter-coder reliability of assigning occupation and activity sector codes while accounting for individual differences and the perceived usability of OPERAS, measured using the System Usability Scale (SUS; range 0-100).
OPERAS usage substantially outperformed manual coding for all coders on both coding time and inter-coder reliability. The median job description coding time was 38 s using OPERAS versus 60.8 s while manually coding. Inter-coder reliability (in Cohen's kappa) ranged 0.61-0.70 and 0.56-0.61 for the PCS, while ranging 0.38-0.61 and 0.34-0.61 for the NAF for OPERAS and manual coding, respectively. The average SUS score was 75.5, indicating good usability.
Compared with manual coding, using OPERAS as DSS for occupational coding improved coding time and inter-coder reliability. Subsequent comparison studies could use OPERAS' ISCO-88 and ISCO-68 classification models. Consequently, OPERAS facilitates large, harmonised job coding in large-scale occupational health research.
对职位描述进行人工编码既耗时又昂贵，还需要专业知识。决策支持系统（DSS）通过提供支持决策的自动建议提供了一种有价值的替代方案，可提高效率，同时允许人工修正以确保可靠性。然而，这一说法尚未在专家编码人员中得到验证。本研究旨在通过使用新的DSS OPERAS将人工编码与决策支持编码进行比较，以填补这一空白。
五名精通使用法国职业分类系统PCS2003和活动部门分类系统NAF2008的专家编码人员，先后分别使用OPERAS和人工方式对CONSTANCES队列中的两个职位描述子集进行编码。随后，我们在考虑个体差异以及使用系统可用性量表（SUS；范围为0 - 100）测量的OPERAS感知可用性的情况下，评估了职业和活动部门代码分配的编码时间和编码人员间的可靠性。
在编码时间和编码人员间的可靠性方面，OPERAS的使用在所有编码人员中均显著优于人工编码。使用OPERAS时，职位描述编码时间的中位数为38秒，而人工编码时为60.8秒。对于PCS，编码人员间的可靠性（以科恩kappa系数衡量）在使用OPERAS时为0.61 - 0.70，人工编码时为0.56 - 0.61；对于NAF，使用OPERAS和人工编码时分别为0.38 - 0.61和0.34 - 0.61。SUS平均得分为75.5，表明可用性良好。
与人工编码相比，使用OPERAS作为职业编码的DSS可提高编码时间和编码人员间的可靠性。后续的比较研究可以使用OPERAS的ISCO - 88和ISCO - 68分类模型。因此，OPERAS有助于在大规模职业健康研究中进行大规模的、统一的职位编码。
