Health Innovation and Evaluation Hub Department, University of Montreal Hospital Research Center (CRCHUM), Pavillon S, 850 Rue Saint-Denis, Montréal QC, Canada.
Ann Work Expo Health. 2022 Jun 6;66(5):551-562. doi: 10.1093/annweh/wxab106.
OBJECTIVES: The use of a job-exposure matrix (JEM) to assess exposure to potential health hazards in occupational epidemiological studies requires coding each participant's job history to a standard occupation and/or industry classification system recognized by the JEM. The objectives of this study were to assess the impact of inter-coder variability in job coding on reliability in exposure estimates derived from linking the job codes to the Canadian job-exposure matrix (CANJEM) and to identify influent parameters. METHOD: Two trained coders independently coded 1000 jobs sampled from a population-based case-control study to the ISCO-1968 occupation classification at the five-digit resolution level, of which 859 could be linked to CANJEM using both assigned codes. Each of the two sets of codes was separately linked to CANJEM and thereby generated, for each of the 258 occupational agents available in CANJEM, two exposure estimates: exposure status (yes/no) and intensity of exposure (low, medium, and high) for exposed jobs only. Then, inter-rater reliability (IRR) was computed (i) after stratifying agents in 4 classes depending, for each, on the proportion of occupation codes in CANJEM defined as 'exposed' and (ii) for two additional scenarios restricted to jobs coded differently: the first one using experts' codes, the other one using codes randomly selected. IRR was computed using Cohen's kappa, PABAK and Gwet's AC1 index for exposure status, and weighted kappa and Gwet's AC2 for exposure intensity. RESULTS: Across all agents and based on all jobs, median (Q1, Q3; Nagents) values were 0.68 (0.59, 0.75; 220) for kappa, 0.99 (0.95, 1.00; 258) for PABAK, and 0.99 (0.97, 1.00; 258) for AC1. For the additional scenarios, median kappa was 0.28 (0.00, 0.45; 209) and -0.01 (-0.02, 00; 233) restricted to jobs coded differently using experts' and random codes, respectively. A similar decreasing pattern was observed for PABAK and AC1 albeit with higher absolute values. Median kappa remained stable across exposure prevalence classes but was more variable for low prevalent agents. PABAK and AC1 decreased with increasing prevalence. Considering exposure intensity and all exposed jobs, median values were 0.79 (0.68, 0.91; 96) for weighted kappa, and 0.95 (0.89, 0.99; 102) for AC2. For the additional scenarios, median kappa was, respectively, 0.28 (-0.04, 0.42) and -0.05 (-0.18, 0.09) restricted to jobs coded differently using experts' and random codes, with a similar though attenuated pattern for AC2. CONCLUSION: Despite reassuring overall reliability results, our study clearly demonstrated the loss of information associated with jobs coded differently. Especially, in cases of low exposure prevalence, efforts should be made to reliably code potentially exposed jobs.
目的:在职业流行病学研究中,使用职业暴露矩阵(JEM)评估潜在健康危害的暴露情况,需要将每个参与者的工作经历编码为 JEM 认可的标准职业和/或行业分类系统。本研究的目的是评估工作编码的编码员间差异对从将工作代码链接到加拿大职业暴露矩阵(CANJEM)中得出的暴露估计值的可靠性的影响,并确定影响参数。
方法:两名经过培训的编码员独立地将从基于人群的病例对照研究中抽取的 1000 个工作编码为 ISCO-1968 职业分类,分辨率为五位数字,其中 859 个工作可以使用两种分配代码与 CANJEM 链接。为每个可用的 258 种职业代理,分别使用两组代码与 CANJEM 链接,从而生成两种暴露估计值:暴露状态(是/否)和暴露工作的暴露强度(低、中、高)。然后,根据每个代理的职业代码在 CANJEM 中定义为“暴露”的比例,将代理分为 4 类(i)计算评分者间可靠性(IRR);(ii)对于两个限制为编码不同的工作的额外情况:第一个使用专家代码,另一个使用随机选择的代码。使用 Cohen's kappa、PABAK 和 Gwet 的 AC1 指数来计算暴露状态的 IRR,使用加权 kappa 和 Gwet 的 AC2 来计算暴露强度的 IRR。
结果:在所有代理和所有工作中,基于所有工作的中位数(Q1、Q3;Nagents)值为 kappa 为 0.68(0.59、0.75;220),PABAK 为 0.99(0.95、1.00;258),AC1 为 0.99(0.97、1.00;258)。对于额外的情况,kappa 的中位数分别为 0.28(0.00、0.45;209)和-0.01(-0.02、00;233),分别限制为使用专家和随机代码的工作编码不同。尽管绝对数值较高,但 PABAK 和 AC1 也表现出类似的递减模式。kappa 的中位数在不同的暴露流行率类别中保持稳定,但在低流行率的代理中更为多变。PABAK 和 AC1 随流行率的增加而降低。考虑到暴露强度和所有暴露的工作,加权 kappa 的中位数为 0.79(0.68、0.91;96),AC2 的中位数为 0.95(0.89、0.99;102)。对于额外的情况,kappa 的中位数分别为 0.28(-0.04、0.42)和-0.05(-0.18、0.09),限制为使用专家和随机代码的工作编码不同,AC2 也存在类似但减弱的模式。
结论:尽管总体可靠性结果令人安心,但我们的研究清楚地表明,与编码不同的工作相关联的信息损失。特别是在低暴露流行率的情况下,应努力可靠地对潜在暴露的工作进行编码。
Ann Work Expo Health. 2022-6-6
Ann Work Expo Health. 2018-8-13
Ann Work Expo Health. 2022-7-2
Commun Med (Lond). 2023-11-4
Int J Environ Res Public Health. 2022-3-13
Cancer Epidemiol Biomarkers Prev. 2021-2
Ann Work Expo Health. 2018-8-13
J Occup Environ Med. 2018-7
Occup Environ Med. 2017-1