Abbott Eduardo F, Serrano Valentina P, Rethlefsen Melissa L, Pandian T K, Naik Nimesh D, West Colin P, Pankratz V Shane, Cook David A
E.F. Abbott is a research fellow, Mayo Clinic Multidisciplinary Simulation Center, Mayo Clinic College of Medicine, Rochester, Minnesota, and adjunct instructor of internal medicine, Department of Internal Medicine, Escuela de Medicina, Pontificia Universidad Catolica de Chile, Santiago, Chile. V.P. Serrano is a research fellow, Knowledge and Evaluation Research Unit, Division of Endocrinology, Diabetes, Metabolism and Nutrition, Mayo Clinic, Rochester, Minnesota, and assistant professor, Department of Nutrition, Diabetes and Metabolism, Escuela de Medicina, Pontificia Universidad Catolica de Chile, Santiago, Chile. M.L. Rethlefsen is deputy director and associate librarian, Spencer S. Eccles Health Sciences Library, and section director, Systematic Review Core, Population Health Research Foundation for Discovery, Center for Clinical and Translational Science, University of Utah, Salt Lake City, Utah. T.K. Pandian is a postgraduate year six resident, Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota. N.D. Naik is a postgraduate year four resident, Department of Surgery, Mayo Clinic College of Medicine, Rochester, Minnesota. C.P. West is professor of medicine, professor of biostatistics, and professor of medical education; associate program director, Internal Medicine Residency Program; and consultant, Division of General Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minnesota. V.S. Pankratz is professor of internal medicine, University of New Mexico Health Sciences Center, Albuquerque, New Mexico. D.A. Cook is professor of medicine and professor of medical education; research chair, Mayo Clinic Multidisciplinary Simulation Center; director of research, Office of Applied Scholarship and Education Science; and consultant, Division of General Internal Medicine, Mayo Clinic College of Medicine, Rochester, Minnesota; ORCID: http://orcid.org/0000-0003-2383-4633.
Acad Med. 2018 Feb;93(2):314-323. doi: 10.1097/ACM.0000000000001773.
To characterize reporting of P values, confidence intervals (CIs), and statistical power in health professions education research (HPER) through manual and computerized analysis of published research reports.
The authors searched PubMed, Embase, and CINAHL in May 2016, for comparative research studies. For manual analysis of abstracts and main texts, they randomly sampled 250 HPER reports published in 1985, 1995, 2005, and 2015, and 100 biomedical research reports published in 1985 and 2015. Automated computerized analysis of abstracts included all HPER reports published 1970-2015.
In the 2015 HPER sample, P values were reported in 69/100 abstracts and 94 main texts. CIs were reported in 6 abstracts and 22 main texts. Most P values (≥77%) were ≤.05. Across all years, 60/164 two-group HPER studies had ≥80% power to detect a between-group difference of 0.5 standard deviations. From 1985 to 2015, the proportion of HPER abstracts reporting a CI did not change significantly (odds ratio [OR] 2.87; 95% CI 1.04, 7.88) whereas that of main texts reporting a CI increased (OR 1.96; 95% CI 1.39, 2.78). Comparison with biomedical studies revealed similar reporting of P values, but more frequent use of CIs in biomedicine. Automated analysis of 56,440 HPER abstracts found 14,867 (26.3%) reporting a P value, 3,024 (5.4%) reporting a CI, and increased reporting of P values and CIs from 1970 to 2015.
P values are ubiquitous in HPER, CIs are rarely reported, and most studies are underpowered. Most reported P values would be considered statistically significant.
通过对已发表研究报告进行人工和计算机分析，描述健康职业教育研究（HPER）中P值、置信区间（CI）和统计功效的报告情况。
作者于2016年5月在PubMed、Embase和CINAHL中检索比较研究。对于摘要和正文的人工分析，他们随机抽取了1985年、1995年、2005年和2015年发表的250篇HPER报告，以及1985年和2015年发表的100篇生物医学研究报告。对摘要的自动化计算机分析包括1970 - 2015年发表的所有HPER报告。
在2015年的HPER样本中，69/100篇摘要和94篇正文中报告了P值。6篇摘要和22篇正文中报告了CI。大多数P值（≥77%）≤0.05。在所有年份中，60/164项两组HPER研究有≥80%的功效检测出组间差异为0.5个标准差。从1985年到2015年，报告CI的HPER摘要比例没有显著变化（优势比[OR]2.87；95%CI 1.04，7.88），而报告CI的正文比例增加（OR 1.96；95%CI 1.39，2.78）。与生物医学研究的比较显示P值报告情况相似，但生物医学中CI的使用更频繁。对56440篇HPER摘要的自动化分析发现，14867篇（26.3%）报告了P值，3024篇（5.4%）报告了CI，并且从1970年到2015年P值和CI的报告有所增加。
P值在HPER中普遍存在，CI很少被报告，并且大多数研究功效不足。大多数报告的P值在统计学上被认为具有显著性。
