Prozesky Detlef R, Kebaetse Masego B, Mogodi Mpho S, Molwantwa Mmoloki C
Department of Medical Education, Faculty of Medicine, University of Botswana, Gaborone, Botswana.
PLoS One. 2024 Dec 31;19(12):e0312730. doi: 10.1371/journal.pone.0312730. eCollection 2024.
A new undergraduate medical programme was instituted at the University of Botswana in 2009. In 2016, the Faculty of Medicine decided to conduct a comprehensive review of the programme. Participants at a planning workshop decided the review had to lead to an in-depth understanding of the programme. The challenge was to develop an evaluation process to achieve this aim.
Standards and theories used in other programme evaluations were investigated. A steering committee developed a 'six-step process' for the review of the programme as a complex system. The process used three evaluation models and various sources of evaluation standards to derive 90 evaluation questions. Quantitative and qualitative data were collected using 96 different instruments, with data triangulation prominently featured. Data analysis used an interpretivist approach. The review process was validated against the Cilliers's ten features of a complex system.
The review process was validated against Cilliers's ten features of a complex system. We found that the 'six-step process' illuminated each of these features in the MBBS programme in turn, and was therefore a valid way of evaluating this programme as a complex system.
In the process of designing and using the 'six-step process' to evaluate a complex medical programme important lessons were learnt: starting the process with complexity theory at the forefront; being as inclusive as possible in data collection; also applying complexity theory to the evaluation of smaller programme components as 'mini-complex systems'; and managing one's inevitable insider bias.
The 'six-step process' as it stands or in adaptation is likely to be useful in similar situations, where evaluators perceive the object of their evaluation to be a complex system, or a component of such a system.
2009年，博茨瓦纳大学设立了一个新的本科医学项目。2016年，医学院决定对该项目进行全面审查。规划研讨会上的参与者认为，此次审查必须深入了解该项目。面临的挑战是制定一个评估流程以实现这一目标。
研究了其他项目评估中使用的标准和理论。一个指导委员会为将该项目作为一个复杂系统进行审查制定了一个“六步流程”。该流程使用了三种评估模型和各种评估标准来源，得出了90个评估问题。使用96种不同工具收集定量和定性数据，数据三角验证是突出特点。数据分析采用解释主义方法。对照西里尔斯提出的复杂系统的十个特征对审查过程进行了验证。
对照西里尔斯提出的复杂系统的十个特征对审查过程进行了验证。我们发现“六步流程”依次阐明了医学学士项目中的每一个特征，因此是将该项目作为一个复杂系统进行评估的有效方式。
在设计和使用“六步流程”评估一个复杂医学项目的过程中，我们吸取了重要经验教训：以复杂性理论为首要出发点开展该流程；在数据收集方面尽可能做到包容；也将复杂性理论应用于对较小项目组成部分作为“微型复杂系统”的评估；以及处理不可避免的内部人偏见。
现有的或经过调整的“六步流程”可能在类似情况下有用，即评估者认为其评估对象是一个复杂系统或此类系统的一个组成部分的情况。
