de Oliveira Filho Getúlio R, Dal Mago Adilson José, Garcia Jorge Hamilton Soares, Goldschmidt Ranulfo
Department of Anesthesiology, Hospital Governador Celso Ramos, Rua Luiz Delfino 111/902-88015360 Florianópolis, SC, Brazil.
Anesth Analg. 2008 Oct;107(4):1316-22. doi: 10.1213/ane.0b013e318182fbdd.
We aimed 1) to develop a valid and reliable instrument for faculty supervision evaluation by anesthesia residents and 2) to disclose the sources of error in residents' ratings.
A qualitative study involving residents and faculty identified constructs of supervisory ability, which were entered as items in a measurement instrument used by 19 residents to evaluate 39 instructors during a 6-mo period. The instrument was psychometrically tested under classical item and generalizability theories. A decision study, using the parameters of the generalizability (G) study, estimated the number of resident ratings needed to produce dependable measures of a single faculty.
Nine dimensions emerged from the qualitative study: planning perianesthesia care, providing feedback ("the instructor provides me timely, informal, non-threatening comments on my performance and shows me ways to improve"); being available ("the instructor is promptly available to help me solve problems with patients and procedures"); giving opportunities/fostering resident autonomy; stimulating patient-based learning; demonstrating professionalism; being present during the critical events; demonstrating interpersonal skills; being concerned about safety. Residents provided 970 evaluations. The instrument exhibited internal consistency (Cronbach's alpha=0.93), content and face validities, and a single-factor structure. Generalizability and dependability coefficients were 0.93. Between-instructors differences accounted for 56% of score variance. Resident-instructor interactions accounted for 44% of score variance, indicating that scores were influenced by each resident's unique perceptions of instructors (halo effect). According to the results of the decision study, dependability of measures within the 75% to 95% range could be expected with 3 to 33 residents rating each faculty member, respectively.
The nine-item instrument produced valid and reliable measures of faculty supervision. However, a significant amount of halo effect biased such measures. G-studies may help identify the type and magnitude of rater biases affecting resident-generated faculty supervision evaluations, and can be useful for interpreting their results, especially if personnel decisions (e.g., tenure, promotion) rely on such measures.
我们旨在1）开发一种由麻醉住院医师对带教教师进行监督评价的有效且可靠的工具，以及2）揭示住院医师评分中的误差来源。
一项涉及住院医师和带教教师的定性研究确定了监督能力的构成要素，并将其作为测量工具中的项目，19名住院医师在6个月的时间里使用该工具对39名带教教师进行评价。该工具在经典项目理论和概化理论下进行了心理测量学测试。一项决策研究利用概化（G）研究的参数，估计了为得出单个带教教师可靠测量结果所需的住院医师评分数量。
定性研究得出了九个维度：围麻醉期护理计划、提供反馈（“带教教师及时、非正式且无威胁地就我的表现给予我评价，并告诉我改进的方法”）、随时可及（“带教教师能及时帮助我解决患者和操作方面的问题”）、给予机会/培养住院医师自主性、激发基于患者的学习、展现专业素养、在关键事件期间在场、展现人际技能、关注安全。住院医师提供了970份评价。该工具显示出内部一致性（克朗巴哈系数=0.93）、内容效度和表面效度，以及单因素结构。概化系数和可靠性系数为0.93。带教教师之间的差异占分数方差的56%。住院医师与带教教师的互动占分数方差的44%，表明分数受到每位住院医师对带教教师的独特看法（晕轮效应）的影响。根据决策研究的结果，分别由3至33名住院医师对每位带教教师进行评分时，可预期测量结果在75%至95%范围内的可靠性。
这个包含九个项目的工具产生了对带教教师监督的有效且可靠的测量结果。然而，大量的晕轮效应使这些测量结果产生偏差。G研究可能有助于识别影响住院医师对带教教师监督评价的评分者偏差类型和程度，并有助于解释其结果，特别是在人事决策（如终身教职、晋升）依赖此类测量结果的情况下。
