Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Copenhagen, Denmark.
Respiration. 2012;83(2):140-6. doi: 10.1159/000332333. Epub 2011 Oct 6.
Several tools have been created to assess competence in bronchoscopy. However, educational guidelines still use an arbitrary number of performed procedures to decide when basic competency is acquired.
The purpose of this study was to define pass/fail scores for two bronchoscopy assessment tools, and investigate how these scores relate to physicians' experience regarding the number of bronchoscopy procedures performed.
We studied two assessment tools and used two standard setting methods to create cut scores: the contrasting-groups method and the extended Angoff method. In the first we compared bronchoscopy performance scores of 14 novices with the scores of 14 experienced consultants to find the score that best discriminated between the two groups. In the second we asked an expert group of 7 experienced bronchoscopists to judge how a borderline trainee would perform on each item of the test.
Using the contrasting-groups method we found a standard that would fail all novices and pass all consultants. A clear pass related to prior experience of 75 procedures. The consequences of using the extended Angoff method were also acceptable: all trainees who had performed less than 50 bronchoscopies failed the test and all consultants passed. A clear pass related to 80 procedures.
Our proposed pass/fail scores for these two methods seem appropriate in terms of consequences. Prior experience with the performance of 75 and 80 bronchoscopies, respectively, seemed to ensure basic competency. In the future objective assessment tools could become an important aid in the certification of physicians performing bronchoscopies.
已有多种工具被开发出来用于评估支气管镜检查的能力。然而，教育指南仍然使用执行程序的任意数量来决定何时获得基本能力。
本研究的目的是定义两种支气管镜检查评估工具的通过/失败分数，并研究这些分数与医生执行支气管镜检查程序数量的经验之间的关系。
我们研究了两种评估工具，并使用两种标准设定方法来创建切割分数：对比组方法和扩展 Angoff 方法。在第一种方法中，我们比较了 14 名新手和 14 名经验丰富的顾问的支气管镜检查表现评分，以找到最佳区分两组的分数。在第二种方法中，我们要求一个由 7 名经验丰富的支气管镜检查医师组成的专家组对每个测试项目进行判断，一个边缘学员的表现会如何。
使用对比组方法，我们发现了一个标准，该标准将使所有新手都不及格，而所有顾问都及格。一个明确的及格分数与 75 次操作经验有关。使用扩展 Angoff 方法的后果也是可以接受的：所有操作少于 50 次的学员都未能通过测试，而所有顾问都通过了。一个明确的及格分数与 80 次操作有关。
就后果而言，我们提出的这两种方法的通过/失败分数似乎是合适的。分别具有 75 次和 80 次支气管镜检查操作的先前经验似乎确保了基本能力。在未来，客观评估工具可能成为对执行支气管镜检查的医生进行认证的重要辅助手段。
