Department of Environment, Climate Change and Health, World Health Organization, Geneva, Switzerland.
Department of Environment, Climate Change and Health, World Health Organization, Geneva, Switzerland.
Environ Int. 2022 Mar;161:107136. doi: 10.1016/j.envint.2022.107136. Epub 2022 Feb 16.
The World Health Organization (WHO) and the International Labour Organization (ILO) have produced the WHO/ILO Joint Estimates of the Work-related Burden of Disease and Injury (WHO/ILO Joint Estimates). For these, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors have been conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methodology is to assess the quality of evidence across studies. In this article, we present the approach applied in these WHO/ILO systematic reviews for performing such assessments on studies of prevalence of exposure. It is called the Quality of Evidence in Studies estimating Prevalence of Exposure to Occupational risk factors (QoE-SPEO) approach. We describe QoE-SPEO's development to date, demonstrate its feasibility reporting results from pilot testing and case studies, note its strengths and limitations, and suggest how QoE-SPEO should be tested and developed further.
Following a comprehensive literature review, and using expert opinion, selected existing quality of evidence assessment approaches used in environmental and occupational health were reviewed and analysed for their relevance to prevalence studies. Relevant steps and components from the existing approaches were adopted or adapted for QoE-SPEO. New steps and components were developed. We elicited feedback from other systematic review methodologists and exposure scientists and reached consensus on the QoE-SPEO approach. Ten individual experts pilot-tested QoE-SPEO. To assess inter-rater agreement, we counted ratings of expected (actual and non-spurious) heterogeneity and quality of evidence and calculated a raw measure of agreement (P) between individual raters and rater teams for the downgrade domains. P ranged between 0.00 (no two pilot testers selected the same rating) and 1.00 (all pilot testers selected the same rating). Case studies were conducted of experiences of QoE-SPEO's use in two WHO/ILO systematic reviews.
We found no existing quality of evidence assessment approach for occupational exposure prevalence studies. We identified three relevant, existing approaches for environmental and occupational health studies of the effect of exposures. Assessments using QoE-SPEO comprise three steps: (1) judge the level of expected heterogeneity (defined as non-spurious variability that can be expected in exposure prevalence, within or between individual persons, because exposure may change over space and/or time), (2) assess downgrade domains, and (3) reach a final rating on the quality of evidence. Assessments are conducted using the same five downgrade domains as the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach: (a) risk of bias, (b) indirectness, (c) inconsistency, (d) imprecision, and (e) publication bias. For downgrade domains (c) and (d), the assessment varies depending on the level of expected heterogeneity. There are no upgrade domains. The QoE-SPEO's ratings are "very low", "low", "moderate", and "high". To arrive at a final decision on the overall quality of evidence, the assessor starts at "high" quality of evidence and for each domain downgrades by one or two levels for serious concerns or very serious concerns, respectively. In pilot tests, there was reasonable agreement in ratings for expected heterogeneity; 70% of raters selected the same rating. Inter-rater agreement ranged considerably between downgrade domains, both for individual rater pairs (range P: 0.36-1.00) and rater teams (0.20-1.00). Sparse data prevented rigorous assessment of inter-rater agreement in quality of evidence ratings.
We present QoE-SPEO as an approach for assessing quality of evidence in prevalence studies of exposure to occupational risk factors. It has been developed to its current version (as presented here), has undergone pilot testing, and was applied in the systematic reviews for the WHO/ILO Joint Estimates. While the approach requires further testing and development, it makes steps towards filling an identified gap, and progress made so far can be used to inform future work in this area.
世界卫生组织（世卫组织）和国际劳工组织（劳工组织）联合编制了《与工作有关的疾病和伤害负担全球估计（世卫组织/劳工组织联合估计）》。为此，对估计某些职业风险因素的暴露率的研究进行了系统审查，以提供估计接触工人人数的数据输入。系统评价方法的一个关键部分是评估研究证据的质量。在本文中，我们介绍了世卫组织/劳工组织这些系统评价中用于对职业风险因素暴露率的研究进行此类评估的方法，称为《职业风险因素暴露率研究证据质量评估》（QoE-SPEO）方法。我们描述了 QoE-SPEO 的发展历程，展示了其在试点测试和案例研究中的可行性报告结果，指出了其优势和局限性，并提出了如何对 QoE-SPEO 进行测试和进一步开发。
在进行全面文献回顾后，我们利用专家意见，对环境和职业健康领域中使用的现有的证据质量评估方法进行了回顾和分析，以评估其与流行率研究的相关性。从现有的方法中选择了相关的步骤和组成部分，并对其进行了采用或改编，以制定 QoE-SPEO。同时还开发了新的步骤和组成部分。我们向其他系统评价方法学家和暴露科学家征求了反馈意见，并就 QoE-SPEO 方法达成了共识。十位独立专家对 QoE-SPEO 进行了试点测试。为了评估评分者间的一致性，我们计算了预期（实际和非虚假）异质性和证据质量的评分，并计算了个人评分者和评分者团队对降级领域的原始一致性评分（P）。P 的范围从 0.00（没有两位试点测试者选择相同的评分）到 1.00（所有试点测试者选择相同的评分）。我们对 QoE-SPEO 在两项世卫组织/劳工组织联合估计中的使用经验进行了案例研究。
我们没有找到针对职业暴露流行率研究的现有证据质量评估方法。我们确定了适用于环境和职业健康领域的暴露效应研究的三个相关的现有方法。使用 QoE-SPEO 进行的评估包括三个步骤：（1）判断预期异质性的水平（定义为由于暴露可能随时间和/或空间而变化，因此可以预期在个体内部或个体之间存在的非虚假变异性，因为暴露可能会发生变化），（2）评估降级领域，以及（3）对证据质量做出最终评级。评估使用与 Grading of Recommendations Assessment, Development and Evaluation（GRADE）方法相同的五个降级领域进行：（a）偏倚风险，（b）间接性，（c）不一致性，（d）不精确性，以及（e）发表偏倚。对于降级领域（c）和（d），评估根据预期异质性的水平而有所不同。没有升级领域。QoE-SPEO 的评分是“极低”、“低”、“中等”和“高”。为了对整体证据质量做出最终决策，评估者从“高”质量证据开始，并根据每个领域的严重关切或非常严重关切，分别降级一到两个级别。在试点测试中，对于预期异质性的评分有合理的一致性；70%的评分者选择了相同的评分。评分者间的一致性在降级领域之间差异很大，无论是在个体评分者对之间（范围 P：0.36-1.00）还是评分者团队之间（0.20-1.00）。由于数据稀疏，无法对质量证据评分的评分者间一致性进行严格评估。
我们提出了 QoE-SPEO 作为评估职业风险因素暴露率研究证据质量的方法。它已经发展到目前的版本（如本文所述），已经进行了试点测试，并在世卫组织/劳工组织联合估计的系统评价中得到了应用。虽然该方法需要进一步测试和开发，但它在填补已确定的空白方面取得了进展，并且迄今为止取得的进展可以为该领域的未来工作提供信息。
