Tonia Thomy, Buitrago-Garcia Diana, Peter Natalie, Mesa-Vieira Cristina, Li Tianjing, Furukawa Toshi A, Cipriani Andrea, Leucht Stefan, Low Nicola, Salanti Georgia
medRxiv. 2023 Feb 2:2023.02.01.23285335. doi: 10.1101/2023.02.01.23285335.
Biases affect how certain we are about the available evidence, however no standard tool for assessing the risk of bias (RoB) in prevalence studies exists. For the purposes of a living systematic review on prevalence of mental health disorders during the COVID-19 pandemic, we developed a RoB tool to evaluate prevalence studies in mental health (RoB-PrevMH) and tested interrater reliability.
We reviewed existing RoB tools for prevalence studies until September 2020, to develop a tool for prevalence studies in mental health. We tested the reliability of assessments by different users of RoB-PrevMH in 83 studies stemming from two systematic reviews of prevalence studies in mental health. We assessed the interrater agreement by calculating the proportion of agreement and Kappa statistic for each item.
RoB-PrevMH consists of three items that address selection bias and information bias. Introductory and signaling questions guide the application of the tool to the review question. The interrater agreement for the three items was 83%, 90% and 93%. The weighted kappa was 0.63 (95% CI 0.54 to 0.73), 0.71 (95% CI 0.67 to 0.85) and 0.32 (95% CI -0.04 to -0.63), respectively.
We developed a brief, user friendly, and adaptable tool for assessing RoB in studies on prevalence of mental health disorders. Initial results for interrater agreement were fair to substantial. The tool's validity, reliability, and applicability should be assessed in future projects.
偏倚会影响我们对现有证据的确信程度，然而目前尚无用于评估患病率研究中偏倚风险（RoB）的标准工具。为了对2019冠状病毒病（COVID-19）大流行期间心理健康障碍的患病率进行实时系统评价，我们开发了一种RoB工具来评估心理健康方面的患病率研究（RoB-PrevMH），并测试了评分者间信度。
我们检索了截至2020年9月的现有患病率研究RoB工具，以开发一种心理健康患病率研究工具。我们在两项心理健康患病率研究系统评价中的83项研究中，测试了不同使用者对RoB-PrevMH评估的可靠性。我们通过计算每项的一致性比例和Kappa统计量来评估评分者间的一致性。
RoB-PrevMH由三个涉及选择偏倚和信息偏倚的项目组成。引导性问题和信号问题指导该工具应用于综述问题。这三个项目的评分者间一致性分别为83%、90%和93%。加权Kappa分别为0.63（95%CI 0.54至0.73）、0.71（95%CI 0.67至0.85）和0.32（95%CI -0.04至 -0.63）。
我们开发了一种简短、用户友好且可调整的工具，用于评估心理健康障碍患病率研究中的RoB。评分者间一致性的初步结果为尚可至较高。该工具的有效性、可靠性和适用性应在未来项目中进行评估。
