Witter Sophie, Kardan Andrew, Scott Molly, Moore Lucie, Shaxson Louise
International Health Financing and Health Systems, Queen Margaret University, Edinburgh and Oxford Policy Management Associate, Oxford, UK.
Oxford Policy Management, Oxford, UK.
Health Res Policy Syst. 2017 Oct 2;15(1):86. doi: 10.1186/s12961-017-0250-4.
The Demand-Driven Evaluations for Decisions (3DE) programme was piloted in Zambia and Uganda in 2012-2015. It aimed to answer evaluative questions raised by policymakers in Ministries of Health, rapidly and with limited resources. The aim of our evaluation was to assess whether the 3DE model was successful in supporting and increasing evidence-based policymaking, building capacity and changing behaviour of Ministry staff.
Using mixed methods, we compared the ex-ante theory of change with what had happened in practice, why and with what results (intended and unintended), including a qualitative assessment of 3DE's contribution. Data sources included a structured quality assessment of the five impact evaluations produced, 46 key informant interviews at national and international levels, structured extraction from 170 programme documents, a wider literature review of relevant topics, and a political economy analysis conducted in Zambia.
We found that 3DE had a very limited contribution to changing evidence-based policymaking, capacity and behaviour in both countries as a result of having a number of aspirations not all compatible with one another. Co-developing evaluation questions was more time-consuming than anticipated, Ministry evidence needs did not fit neatly into questions suitable for impact evaluations and constricted timeframes for undertaking trials did not necessarily produce the most effective results and value for money. The evaluation recommended a focusing of objectives and a more strategic approach to strengthening evaluative demand and capacity.
Lessons emerge that are likely to apply in other low- and middle-income settings, such as the importance of supporting evaluative thinking and capacity within wider institutions, of understanding the political economy of evidence use and its uptake, and of allowing for some flexibility in terms of programme targets. Fixating on one type of evidence is unhelpful in the context of institutions like ministries of health, which require a wide range of evidence to plan and deliver programmes. In addition, having success tied to indicators, such as number of 'policy decisions made', provides potentially perverse incentives and neglects arguably more important aspects such as incremental programmatic adjustments and improved implementation.
2012年至2015年期间，在赞比亚和乌干达开展了决策需求驱动评估（3DE）项目试点。该项目旨在利用有限资源快速回答卫生部政策制定者提出的评估问题。我们评估的目的是评估3DE模式是否成功支持并促进了循证决策，建设了能力并改变了卫生部工作人员的行为。
我们采用混合方法，将事前变革理论与实际发生的情况、原因及结果（预期和非预期）进行比较，包括对3DE贡献的定性评估。数据来源包括对所产生的五项影响评估进行的结构化质量评估、在国家和国际层面进行的46次关键信息访谈、从170份项目文件中进行的结构化提取、对相关主题的更广泛文献综述，以及在赞比亚进行的政治经济分析。
我们发现，由于3DE存在一些相互不兼容的目标，它对改变两国的循证决策、能力和行为的贡献非常有限。共同制定评估问题比预期更耗时，卫生部的证据需求与适合影响评估的问题不完全匹配，而且开展试验的时间框架受限不一定能产生最有效的结果和性价比。评估建议明确目标，并采取更具战略性的方法来加强评估需求和能力。
从中得出的经验教训可能适用于其他低收入和中等收入环境，例如在更广泛的机构中支持评估思维和能力的重要性、理解证据使用及其采纳的政治经济学，以及在项目目标方面允许一定灵活性的重要性。在卫生部这样的机构中，只专注于一种证据是没有帮助的，因为这些机构需要广泛的证据来规划和实施项目。此外，将成功与指标挂钩，如“做出的政策决策数量”，可能会产生适得其反的激励措施，并忽视了诸如渐进式项目调整和改进实施等更重要的方面。
