Dandona Lalit, Raban Magdalena Z, Dandona Rakhi
Public Health Foundation of India, ISID Campus, 4 Institutional Area, Vasant Kunj, New Delhi 110070, India.
Natl Med J India. 2011 Sep-Oct;24(5):263-8.
BACKGROUND. Analysis of the scope and quality of evaluations of health system/policy interventions done in India is not available. Such analysis can help in conducting more useful evaluations. METHODS. We accessed evaluation reports of health system/ policy interventions aimed at improving population health in India, reported during 2001-08, which were available in the public domain through extensive internet searches. We developed and used a classification system for the type of evaluation, commissioning agency, health system/policy area covered and methodology used, and a method for assessing the quality of evaluation reports. RESULTS. Of the 219 total evaluation reports in the public domain, 6% assessed needs, 22% process, 42% outcome and 30% impact. Seventy-six per cent evaluations were commissioned by international agencies. Among health system components, services were the focus of evaluation in 74.9% of reports, with human resources, financing, drugs/products, information system and governance having little representation. Only 21% of evaluation reports were rated as good quality. Among evaluations based mainly on health system data, 42% were poor quality compared with 20% that were based on population data. Seventy-two per cent of the outcome/impact evaluations presented only basic tabulations and just 12% attempted multivariate analysis. Eighty-two per cent of the outcome/impact evaluations had no controls, among which 42% were poor quality versus 17% poor quality among outcome/impact evaluations with controls. Among the 54% evaluations in which the intervention implementer was involved, only 1% reported negative conclusion about the intervention compared with 37% among evaluations in which the implementer was not involved. CONCLUSION. This analysis of health system/policy intervention evaluation reports from India identifies specific areas that need improvement. We recommend that Indian agencies should commission more evaluations as international agencies currently dominate, involvement of intervention implementer in the evaluation needs scrutiny as potential bias is suggested by our analysis, and health system components other than services need more attention. Outcome/impact evaluations need to incorporate controls in design and multivariate techniques more often in their analysis to achieve higher quality robust evaluations in India.
背景。目前尚无对印度开展的卫生系统/政策干预评估的范围和质量的分析。此类分析有助于开展更有用的评估。方法。我们通过广泛的互联网搜索，获取了2001年至2008年期间公开的、旨在改善印度人口健康的卫生系统/政策干预评估报告。我们开发并使用了一个分类系统，用于评估类型、委托机构、所涵盖的卫生系统/政策领域以及所使用的方法，并使用一种方法来评估评估报告的质量。结果。在公开的219份评估报告中，6%评估需求，22%评估过程，42%评估结果，30%评估影响。76%的评估由国际机构委托。在卫生系统组成部分中，74.9%的报告将服务作为评估重点，人力资源、筹资、药品/产品、信息系统和治理方面的评估较少。只有21%的评估报告被评为高质量。在主要基于卫生系统数据的评估中，42%质量较差，而基于人群数据的评估中这一比例为20%。72%的结果/影响评估仅呈现基本表格，只有12%尝试进行多变量分析。82%的结果/影响评估没有对照组，其中42%质量较差，而有对照组的结果/影响评估中质量较差的比例为17%。在干预实施者参与的54%的评估中，只有1%报告了关于干预的负面结论，而在实施者未参与的评估中这一比例为37%。结论。对印度卫生系统/政策干预评估报告的这一分析确定了需要改进的具体领域。我们建议，由于目前国际机构占主导地位，印度机构应委托开展更多评估；鉴于我们的分析表明存在潜在偏差，干预实施者参与评估需要受到审查；除服务外的卫生系统组成部分需要更多关注。结果/影响评估需要在设计中纳入对照组，并在分析中更多地使用多变量技术，以在印度实现更高质量的稳健评估。
