From the Department of Anaesthesia, ITU and Pain Management, Mater Dei Hospital, Msida, Malta (SS), Department of Anaesthesiology, Erasmus University Medical Centre, Rotterdam, the Netherlands (MK), European Society of Anaesthesiology and Intensive Care, Brussels, Belgium (MK, BA, HS, RDL, JBE), Department of Anaesthesia, University Hospital of Wales, Cardiff, UK (BA), Institute for Medical Education, University of Bern, Bern, Switzerland (JBE), CINTESIS@RISE - Centre for Health Technology and Services Research, Porto, Portugal (JBE) and Institute of Anaesthesiology and Intensive Care, Salemspital, Hirslanden Medical Group, Bern, Switzerland (JBE).
Eur J Anaesthesiol. 2024 Dec 1;41(12):921-931. doi: 10.1097/EJA.0000000000002052. Epub 2024 Aug 28.
The European Diploma of Anaesthesiology and Intensive Care (EDAIC) Part II examination is a supranational examination for anaesthesiologists.
We explore the impact of examiner pairing on leniency and stringency, commonly referred to as the 'hawk-dove effect'. We investigate the potential variations in grading approaches, resulting from different examiner pairs and their implications for candidate performance.
Retrospective cohort, observational design.
EDAIC Part II examination data from 2021 to 2023.
Three hundred and twenty-five examiners across 122 EDAIC Part II single-day examination sessions.
We analysed the influence of examiner leniency and examiner pairing on candidate performance in the EDAIC Part II using many-facet Rasch modelling.
The study's main outcome measure was determining a leniency score among the examiner population. The study also aimed to assess how examiner pairing influenced candidate performance, as measured by their scores in the examination.
During the study period, the number of examiners who participated in 2021, 2022 and 2023 were 253, 242 and 247, respectively. The median [IQR] single-day sessions attended were 7.0 [3 to 10]. The examination data revealed a mean leniency score of 0 (95% confidence interval (CI) -0.046 to 0.046), with the standard deviation being one-third that of the candidates' ability scores. There were 1424 different pairs of examiners, with most pairs (97%) having only a one-point difference in marking. The mean leniency score for the pair of examiners was -0.053 (95% CI -0.069 to -0.037).
The variations in grading approaches associated with different pairings emphasise the potential for the 'hawk-dove effect' to influence candidate performance and outcomes. Understanding these variations can guide curriculum development, examiner training and coupling, ensuring a balanced and equitable assessment process.
None.
欧洲麻醉学和重症监护文凭（EDAIC）第二部分考试是一项针对麻醉师的跨国考试。
我们探讨了考官配对对宽松度和严格度的影响，通常称为“鹰鸽效应”。我们研究了由于不同的考官对评分方法的潜在变化，以及这些变化对考生表现的影响。
回顾性队列，观察性设计。
2021 年至 2023 年 EDAIC 第二部分考试数据。
122 次 EDAIC 第二部分单日考试中共有 325 名考官。
我们使用多方面 Rasch 模型分析了考官的宽松度和考官配对对 EDAIC 第二部分考生表现的影响。
该研究的主要观察指标是确定考官群体中的宽松度评分。该研究还旨在评估考官配对如何影响考生的表现，这可以通过他们在考试中的得分来衡量。
在研究期间，2021 年、2022 年和 2023 年参加考试的考官人数分别为 253 人、242 人和 247 人。中位数[IQR]参加的单日考试为 7.0[3 至 10]。考试数据显示，平均宽松度评分为 0（95%置信区间[CI]为-0.046 至 0.046），其标准差为考生能力得分的三分之一。共有 1424 对不同的考官，其中大多数考官（97%）的评分仅相差一分。考官对的平均宽松度评分为-0.053（95%CI-0.069 至-0.037）。
与不同配对相关的评分方法的变化强调了“鹰鸽效应”可能影响考生表现和结果的潜力。了解这些变化可以指导课程开发、考官培训和配对，确保评估过程平衡和公平。
无。
