Alizadeh Maryam, Rahmati Rahem, Zarimeidani Fatemeh, Hasani Fatemeh, Ghaedi Arshin, Bazrgar Aida, Hosseini Doalame Reza, Vahedi Hojat, Hekmat Hamidreza, Omidi Negar
Department of Medical Education, School of Medicine, Health Professions Education Research Center, Tehran University of Medical Sciences, Tehran, Iran.
Students Research Committee, Shahrekord University of Medical Sciences, Shahrekord, Iran.
Medicine (Baltimore). 2025 May 2;104(18):e42316. doi: 10.1097/MD.0000000000042316.
Even though there has been a lot of research in medical education, the quality of it has not increased similarly. This study aimed to provide a valid and reliable user-friendly tool for evaluating search strategies in medical education systematic reviews. This mixed study was conducted in 2019 to 2021, including 3 phases: systematic search, developing a medical education research quality (MERSQ) checklist, and evaluation of the search quality of best evidence in medical education collaboration (BEME) and non-BEME reviews. Three hundred nineteen items were retrieved from the systematic search of PubMed, Embase, Scopus, Psychinfo, ERIC, and Google Scholar. Following ensuring acceptable criteria, 30 items were included in comprehensiveness or reproducibility guarantees. The results showed that the instrument had an the intra-class correlation coefficient of 0.922 (P = .002), the reproducibility guarantee had 0.903 (P = .003), and the comprehensiveness guarantee had 0.926 (P = .006). We also calculated inter-rater reliability and internal consistency using Cronbach alpha of 0.827 (P < .001) and an instrument the intra-class correlation coefficient of 0.978. Using MERSQ, the overall search quality (41.75 vs 31.25, P = .009), reproducibility (22 vs 14.50, P = .004), and comprehensive score (18.75 vs 15.75, P = .880) of BEME studies were higher than non-BEME ones. Moreover, we found only 30% of studies completed searching documents. The search strategy query concerning the selection of synonym terms received the lowest score among studies. This study led to the development of a valid and reliable checklist for evaluating the search quality of medical education systematic reviews. Utilizing the MERSQ checklist, we found that BEME studies had higher quality than non-BEME ones, making the results from BEME studies more reliable.
尽管医学教育领域已经开展了大量研究，但其质量并未相应提高。本研究旨在提供一种有效、可靠且用户友好的工具，用于评估医学教育系统评价中的检索策略。这项混合研究于2019年至2021年进行，包括三个阶段：系统检索、制定医学教育研究质量（MERSQ）清单，以及评估医学教育最佳证据协作（BEME）和非BEME评价的检索质量。通过对PubMed、Embase、Scopus、Psychinfo、ERIC和谷歌学术进行系统检索，共检索到319项。在确保符合可接受标准后，30项被纳入全面性或可重复性保证。结果显示，该工具的组内相关系数为0.922（P = .002），可重复性保证为0.903（P = .003），全面性保证为0.926（P = .006）。我们还使用Cronbach α系数0.827（P < .001）计算了评分者间信度和内部一致性，以及工具的组内相关系数0.978。使用MERSQ，BEME研究的总体检索质量（41.75对31.25，P = .009）、可重复性（22对14.50，P = .004）和综合评分（18.75对15.75，P = .880）均高于非BEME研究。此外，我们发现只有30%的研究完成了文献检索。在各项研究中，关于同义词项选择的检索策略查询得分最低。本研究促成了一份有效且可靠的清单的开发，用于评估医学教育系统评价的检索质量。利用MERSQ清单，我们发现BEME研究的质量高于非BEME研究，这使得BEME研究的结果更可靠。
