Jabeen Sumera
The University of Melbourne, Australia.
Eval Program Plann. 2018 Jun;68:262-274. doi: 10.1016/j.evalprogplan.2017.09.005. Epub 2017 Sep 18.
Social development programmes are deliberate attempts to bring about change and unintended outcomes can be considered as inherent to any such intervention. There is now a solid consensus among the international evaluation community regarding the need to consider unintended outcomes as a key aspect in any evaluative study. However, this concern often equates to nothing more than false piety. Exiting evaluation theory suffers from overlap of terminology, inadequate categorisation of unintended outcomes and lack of guidance on how to study them. To advance the knowledge of evaluation theory, methods and practice, the author has developed an evaluation approach to study unintended effects using a theory building, testing and refinement process. A comprehensive classification of unintended outcomes on the basis of knowability, value, distribution and temporality helped specify various type of unintended outcomes for programme evaluation. Corresponding to this classification, a three-step evaluation process was proposed including a) outlining programme intentions b) forecasting likely unintended effects c) mapping the anticipated and understanding unanticipated unintended outcomes. This unintended outcomes evaluation approach (UOEA) was then trialled by undertaking a multi-site and multi-method case study of a poverty alleviation programme in Pakistan and refinements were made to the approach.The case study revealed that this programme was producing a number of unintended effects, mostly negative, affecting those already disadvantaged such as the poorest, women and children. The trialling process demonstrated the effectiveness of the UOEA and suggests that this can serve as a useful guide for future evaluation practice. It also provides the discipline of evaluation with an empirically-based reference point for further theoretical developments in the study of unintended outcomes.
社会发展项目是为带来变革而进行的刻意尝试,意外结果可被视为任何此类干预固有的一部分。目前,国际评估界已就将意外结果视为任何评估研究的关键方面达成了坚实共识。然而,这种关注往往只不过是虚假的虔诚。现有的评估理论存在术语重叠、意外结果分类不充分以及缺乏关于如何研究这些结果的指导等问题。为了推进评估理论、方法和实践方面的知识,作者开发了一种评估方法,通过理论构建、测试和完善过程来研究意外效果。基于可知性、价值、分布和时间性对意外结果进行的全面分类,有助于明确项目评估中各种类型的意外结果。与此分类相对应,提出了一个三步评估过程,包括:a)概述项目意图;b)预测可能的意外效果;c)描绘预期的意外结果并理解意外的意外结果。然后,通过对巴基斯坦一个扶贫项目进行多地点、多方法的案例研究,对这种意外结果评估方法(UOEA)进行了试验,并对该方法进行了完善。案例研究表明,该项目产生了一些意外效果,大多是负面的,影响到了那些已经处于不利地位的群体,如最贫困者、妇女和儿童。试验过程证明了UOEA的有效性,并表明它可以为未来的评估实践提供有用的指导。它还为评估学科提供了一个基于实证的参考点,以促进意外结果研究方面的进一步理论发展。