文献检索文档翻译深度研究
Suppr Zotero 插件Zotero 插件
邀请有礼套餐&价格历史记录

新学期,新优惠

限时优惠:9月1日-9月22日

30天高级会员仅需29元

1天体验卡首发特惠仅需5.99元

了解详情
不再提醒
插件&应用
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
高级版
套餐订阅购买积分包
AI 工具
文献检索文档翻译深度研究
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2025

评估更新后的 SOCcer v2 算法在三项流行病学研究中对自由文本工作描述进行编码的效果。

Evaluation of the updated SOCcer v2 algorithm for coding free-text job descriptions in three epidemiologic studies.

机构信息

Occupational and Environmental Epidemiology Branch, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD, United States.

Data Science and Engineering Research Group, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD, United States.

出版信息

Ann Work Expo Health. 2023 Jul 6;67(6):772-783. doi: 10.1093/annweh/wxad020.


DOI:10.1093/annweh/wxad020
PMID:37071789
原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC10324641/
Abstract

OBJECTIVES: Computer-assisted coding of job descriptions to standardized occupational classification codes facilitates evaluating occupational risk factors in epidemiologic studies by reducing the number of jobs needing expert coding. We evaluated the performance of the 2nd version of SOCcer, a computerized algorithm designed to code free-text job descriptions to US SOC-2010 system based on free-text job titles and work tasks, to evaluate its accuracy. METHODS: SOCcer v2 was updated by expanding the training data to include jobs from several epidemiologic studies and revising the algorithm to account for nonlinearity and incorporate interactions. We evaluated the agreement between codes assigned by experts and the highest scoring code (a measure of confidence in the algorithm-predicted assignment) from SOCcer v1 and v2 in 14,714 jobs from three epidemiology studies. We also linked exposure estimates for 258 agents in the job-exposure matrix CANJEM to the expert and SOCcer v2-assigned codes and compared those estimates using kappa and intraclass correlation coefficients. Analyses were stratified by SOCcer score, score distance between the top two scoring codes from SOCcer, and features from CANJEM. RESULTS: SOCcer's v2 agreement at the 6-digit level was 50%, compared to 44% in v1, and was similar for the three studies (38%-45%). Overall agreement for v2 at the 2-, 3-, and 5-digit was 73%, 63%, and 56%, respectively. For v2, median ICCs for the probability and intensity metrics were 0.67 (IQR 0.59-0.74) and 0.56 (IQR 0.50-0.60), respectively. The agreement between the expert and SOCcer assigned codes linearly increased with SOCcer score. The agreement also improved when the top two scoring codes had larger differences in score. CONCLUSIONS: Overall agreement with SOCcer v2 applied to job descriptions from North American epidemiologic studies was similar to the agreement usually observed between two experts. SOCcer's score predicted agreement with experts and can be used to prioritize jobs for expert review.

摘要

目的:通过减少需要专家编码的工作数量,将工作描述计算机辅助编码为标准化职业分类代码,便于在流行病学研究中评估职业危险因素。我们评估了 2 版 SOCcer 的性能,该算法是一种基于工作标题和工作任务将自由文本工作描述编码为美国 SOC-2010 系统的计算机算法,以评估其准确性。

方法:通过扩大训练数据,包括来自几项流行病学研究的工作,以及修改算法以考虑非线性和纳入交互作用,更新了 SOCcer v2。我们评估了专家分配的代码与 SOCcer v1 和 v2 的最高分代码(算法预测分配的置信度度量)之间的一致性,该一致性来自三个流行病学研究的 14714 个工作。我们还将职业暴露矩阵 CANJEM 中 258 种暴露剂的暴露估计值与专家和 SOCcer v2 分配的代码联系起来,并使用kappa 和组内相关系数比较了这些估计值。分析按 SOCcer 得分、SOCcer 得分最高的两个代码之间的得分距离以及 CANJEM 的特征进行分层。

结果:SOCcer v2 在 6 位数水平上的一致性为 50%,而 v1 为 44%,三个研究(38%-45%)相似。v2 在 2 位数、3 位数和 5 位数的总体一致性分别为 73%、63%和 56%。对于 v2,概率和强度指标的中位数 ICC 分别为 0.67(IQR 0.59-0.74)和 0.56(IQR 0.50-0.60)。专家与 SOCcer 分配代码之间的一致性随着 SOCcer 得分线性增加。当得分最高的两个代码之间的差异较大时,一致性也会提高。

结论:应用于来自北美的流行病学研究的工作描述的 SOCcer v2 与通常在两位专家之间观察到的一致性相似。SOCcer 的分数可预测与专家的一致性,可用于为专家审查分配工作。

相似文献

[1]
Evaluation of the updated SOCcer v2 algorithm for coding free-text job descriptions in three epidemiologic studies.

Ann Work Expo Health. 2023-7-6

[2]
Computer-based coding of free-text job descriptions to efficiently identify occupations in epidemiological studies.

Occup Environ Med. 2016-6

[3]
Impact of Variability in Job Coding on Reliability in Exposure Estimates Obtained via a Job-Exposure Matrix.

Ann Work Expo Health. 2022-6-6

[4]
Beyond crosswalks: reliability of exposure assessment following automated coding of free-text job descriptions for occupational epidemiology.

Ann Occup Hyg. 2014-5

[5]
Evaluation of Automatically Assigned Job-Specific Interview Modules.

Ann Occup Hyg. 2016-8

[6]
Occupational self-coding and automatic recording (OSCAR): a novel web-based tool to collect and code lifetime job histories in large population-based studies.

Scand J Work Environ Health. 2017-3-1

[7]
Automated Coding of Job Descriptions From a General Population Study: Overview of Existing Tools, Their Application and Comparison.

Ann Work Expo Health. 2023-6-6

[8]
Efficiency of autocoding programs for converting job descriptors into standard occupational classification (SOC) codes.

Am J Ind Med. 2018-12-5

[9]
Beware the Grizzlyman: A comparison of job- and industry-based noise exposure estimates using manual coding and the NIOSH NIOCCS machine learning algorithm.

J Occup Environ Hyg. 2022-7

[10]
JEMs and incompatible occupational coding systems: effect of manual and automatic recoding of job codes on exposure assignment.

Ann Occup Hyg. 2013-1

引用本文的文献

[1]
OPERAS decision support system versus manual job coding: a quantitative analysis on coding time and inter-coder reliability.

Occup Environ Med. 2025-7-9

[2]
Invited Perspective: How Far Have We Come? Revisiting a 2009 Report on Occupational Cancer Research Recommendations.

Environ Health Perspect. 2023-10

本文引用的文献

[1]
Determining occupation for National Violent Death Reporting System records: An evaluation of autocoding programs.

Am J Ind Med. 2021-12

[2]
Occupation Coding of Job Titles: Iterative Development of an Automated Coding Algorithm for the Canadian National Occupation Classification (ACA-NOC).

JMIR Form Res. 2020-8-5

[3]
Efficiency of autocoding programs for converting job descriptors into standard occupational classification (SOC) codes.

Am J Ind Med. 2018-12-5

[4]
Development of a Coding and Crosswalk Tool for Occupations and Industries.

Ann Work Expo Health. 2018-8-13

[5]
Development of and Selected Performance Characteristics of CANJEM, a General Population Job-Exposure Matrix Based on Past Expert Assessments of Exposure.

Ann Work Expo Health. 2018-8-13

[6]
Availability of a New Job-Exposure Matrix (CANJEM) for Epidemiologic and Occupational Medicine Purposes.

J Occup Environ Med. 2018-7

[7]
Capture and coding of industry and occupation measures: Findings from eight National Program of Cancer Registries states.

Am J Ind Med. 2017-8

[8]
Occupational self-coding and automatic recording (OSCAR): a novel web-based tool to collect and code lifetime job histories in large population-based studies.

Scand J Work Environ Health. 2017-3-1

[9]
Occupational exposure to chlorinated solvents and kidney cancer: a case-control study.

Occup Environ Med. 2017-3

[10]
Computer-based coding of free-text job descriptions to efficiently identify occupations in epidemiological studies.

Occup Environ Med. 2016-6

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

推荐工具

医学文档翻译智能文献检索