Department of Palliative and Supportive Services, Flinders University, South Australia, Australia.
BMC Med Res Methodol. 2011 Jan 28;11:12. doi: 10.1186/1471-2288-11-12.
Heart failure is a highly debilitating syndrome with a poor prognosis primarily affecting the elderly. Clinicians wanting timely access to heart failure evidence to provide optimal patient care can face many challenges in locating this evidence. This study developed and validated a search filter of high clinical utility for the retrieval of heart failure articles in OvidSP Medline.
A Clinical Advisory Group was established to advise study investigators. The study set of 876 relevant articles from four heart failure clinical practice guidelines was divided into three datasets: a Term Identification Set, a Filter Development Set, and a Filter Validation Set. A further validation set (the Cochrane Validation Set) was formed using studies included in Cochrane heart failure systematic reviews. Candidate search terms were identified via word frequency analysis. The filter was developed by creating combinations of terms and recording their performance in retrieving items from the Filter Development Set. The filter's recall was then validated in both the Filter Validation Set and the Cochrane Validation Set. A precision estimate was obtained post-hoc by running the filter in Medline and screening the first 200 retrievals for relevance to heart failure.
The four-term filter achieved a recall of 96.9% in the Filter Development Set; 98.2% in the Filter Validation Set; and 97.8% in the Cochrane Validation Set. Of the first 200 references retrieved by the filter when run in Medline, 150 were deemed relevant and 50 irrelevant. The post-hoc precision estimate was therefore 75%.
This study describes an objective method for developing a validated heart failure filter of high recall performance and then testing its precision post-hoc. Clinical practice guidelines were found to be a feasible alternative to hand searching in creating a gold standard for filter development. Guidelines may be especially appropriate given their clinical utility. A validated heart failure filter is now available to support health professionals seeking reliable and efficient access to the heart failure literature.
心力衰竭是一种高度虚弱的综合征,预后不良,主要影响老年人。临床医生希望及时获得心力衰竭的证据,以提供最佳的患者护理,但在寻找这些证据时可能会面临许多挑战。本研究开发并验证了一种在 OvidSP Medline 中检索心力衰竭文章的高临床实用性搜索过滤器。
成立了一个临床顾问小组为研究调查人员提供建议。研究组共 876 篇与四项心力衰竭临床实践指南相关的文章被分为三组:术语识别组、过滤器开发组和过滤器验证组。进一步的验证组(Cochrane 验证组)由纳入 Cochrane 心力衰竭系统评价的研究组成。通过词频分析确定候选搜索词。通过创建术语组合并记录它们在从过滤器开发组中检索项目的性能来开发过滤器。然后在过滤器验证组和 Cochrane 验证组中验证过滤器的召回率。事后通过在 Medline 中运行过滤器并筛选前 200 个检索结果来评估相关性来获得精度估计。
在过滤器开发组中,四术语过滤器的召回率为 96.9%;在过滤器验证组中为 98.2%;在 Cochrane 验证组中为 97.8%。在 Medline 中运行过滤器时检索到的前 200 个参考文献中,有 150 个被认为与心力衰竭相关,50 个不相关。因此,事后的精度估计值为 75%。
本研究描述了一种客观的方法,用于开发具有高召回率性能的验证过的心力衰竭过滤器,然后事后测试其精度。临床实践指南被发现是一种替代手动搜索的可行方法,可用于为过滤器开发创建黄金标准。鉴于其临床实用性,指南可能尤其合适。现在有一个经过验证的心力衰竭过滤器,可以帮助卫生专业人员可靠、高效地获取心力衰竭文献。