Bao Hongchang, Baker Christopher J O, Adisesh Anil
Department of Computer Science, Faculty of Science, Applied Science and Engineering, University of New Brunswick, Saint John, NB, Canada.
Department of Computing Science, University of Alberta, Edmonton, AB, Canada.
JMIR Form Res. 2020 Aug 5;4(8):e16422. doi: 10.2196/16422.
In many research studies, the identification of social determinants is an important activity, in particular, information about occupations is frequently added to existing patient data. Such information is usually solicited during interviews with open-ended questions such as "What is your job?" and "What industry sector do you work in?" Before being able to use this information for further analysis, the responses need to be categorized using a coding system, such as the Canadian National Occupational Classification (NOC). Manual coding is the usual method, which is a time-consuming and error-prone activity, suitable for automation.
This study aims to facilitate automated coding by introducing a rigorous algorithm that will be able to identify the NOC (2016) codes using only job title and industry information as input. Using manually coded data sets, we sought to benchmark and iteratively improve the performance of the algorithm.
We developed the ACA-NOC algorithm based on the NOC (2016), which allowed users to match NOC codes with job and industry titles. We employed several different search strategies in the ACA-NOC algorithm to find the best match, including exact search, minor exact search, like search, near (same order) search, near (different order) search, any search, and weak match search. In addition, a filtering step based on the hierarchical structure of the NOC data was applied to the algorithm to select the best matching codes.
The ACA-NOC was applied to over 500 manually coded job and industry titles. The accuracy rate at the four-digit NOC code level was 58.7% (332/566) and improved when broader job categories were considered (65.0% at the three-digit NOC code level, 72.3% at the two-digit NOC code level, and 81.6% at the one-digit NOC code level).
The ACA-NOC is a rigorous algorithm for automatically coding the Canadian NOC system and has been evaluated using real-world data. It allows researchers to code moderate-sized data sets with occupation in a timely and cost-efficient manner such that further analytics are possible. Initial assessments indicate that it has state-of-the-art performance and is readily extensible upon further benchmarking on larger data sets.
在许多研究中，识别社会决定因素是一项重要活动，特别是关于职业的信息经常被添加到现有的患者数据中。此类信息通常在访谈中通过开放式问题收集，例如“你的工作是什么？”以及“你从事哪个行业？”在能够将这些信息用于进一步分析之前，需要使用编码系统（如加拿大国家职业分类（NOC））对回答进行分类。手动编码是常用方法，这是一项耗时且容易出错的活动，适合自动化。
本研究旨在通过引入一种严格的算法来促进自动编码，该算法仅使用职位名称和行业信息作为输入就能识别NOC（2016）代码。我们使用手动编码的数据集来对算法的性能进行基准测试并迭代改进。
我们基于NOC（2016）开发了ACA-NOC算法，该算法允许用户将NOC代码与职位和行业名称进行匹配。我们在ACA-NOC算法中采用了几种不同的搜索策略来找到最佳匹配，包括精确搜索、轻微精确搜索、相似搜索、近邻（相同顺序）搜索、近邻（不同顺序）搜索、任意搜索和弱匹配搜索。此外，基于NOC数据的层次结构的过滤步骤被应用于该算法以选择最佳匹配代码。
ACA-NOC应用于500多个手动编码的职位和行业名称。在四位数NOC代码级别，准确率为58.7%（332/566），当考虑更宽泛的职业类别时准确率有所提高（三位数NOC代码级别为65.0%，两位数NOC代码级别为72.3%，一位数NOC代码级别为81.6%）。
ACA-NOC是一种用于自动编码加拿大NOC系统的严格算法，并且已使用实际数据进行了评估。它允许研究人员以及时且经济高效的方式对包含职业信息的中等规模数据集进行编码，从而能够进行进一步的分析。初步评估表明它具有先进的性能，并且在对更大数据集进行进一步基准测试时易于扩展。
