Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland.
BMJ Ment Health. 2023 Oct;26(1). doi: 10.1136/bmjment-2023-300694.
There is no standard tool for assessing risk of bias (RoB) in prevalence studies. For the purposes of a living systematic review during the COVID-19 pandemic, we developed a tool to evaluate RoB in studies measuring the prevalence of mental health disorders (RoB-PrevMH) and tested inter-rater reliability.
We decided on items and signalling questions to include in RoB-PrevMH through iterative discussions. We tested the reliability of assessments by different users with two sets of prevalence studies. The first set included a random sample of 50 studies from our living systematic review. The second set included 33 studies from a systematic review of the prevalence of post-traumatic stress disorders, major depression and generalised anxiety disorder. We assessed the inter-rater agreement by calculating the proportion of agreement and Kappa statistic for each item.
RoB-PrevMH consists of three items that address selection bias and information bias. Introductory and signalling questions guide the application of the tool to the review question. The inter-rater agreement for the three items was 83%, 90% and 93%. The weighted kappa scores were 0.63 (95% CI 0.54 to 0.73), 0.71 (95% CI 0.67 to 0.85) and 0.32 (95% CI -0.04 to 0.63), respectively.
RoB-PrevMH is a brief, user-friendly and adaptable tool for assessing RoB in studies on prevalence of mental health disorders. Initial results for inter-rater agreement were fair to substantial. The tool's validity, reliability and applicability should be assessed in future projects.
目前尚无评估流行性病学研究偏倚风险（RoB）的标准工具。为在 COVID-19 大流行期间进行实时系统评价，我们开发了一种用于评估精神健康障碍患病率研究的 RoB（RoB-PrevMH）的工具，并测试了其信度。
我们通过反复讨论决定将 RoB-PrevMH 的纳入项目和信号问题。我们通过两组患病率研究来测试不同使用者评估的可靠性。第一组包括我们实时系统评价中的 50 项随机研究。第二组包括创伤后应激障碍、重度抑郁症和广泛性焦虑障碍患病率系统评价中的 33 项研究。我们通过计算每个项目的一致性比例和 Kappa 统计量来评估组内一致性。
RoB-PrevMH 由三个评估选择偏倚和信息偏倚的项目组成。引言和信号问题指导该工具在审查问题中的应用。三个项目的组内一致性分别为 83%、90%和 93%。加权 Kappa 评分分别为 0.63（95%CI 0.54-0.73）、0.71（95%CI 0.67-0.85）和 0.32（95%CI -0.04-0.63）。
RoB-PrevMH 是一种用于评估精神健康障碍患病率研究 RoB 的简洁、用户友好且可适应的工具。初始组内一致性结果为适度至极好。该工具的有效性、可靠性和适用性应在未来的项目中进行评估。
