Department of Otolaryngology-Head and Neck Surgery, Johns Hopkins University, Baltimore, Maryland, USA.
Laryngoscope. 2012 Nov;122(11):2418-21. doi: 10.1002/lary.23494. Epub 2012 Aug 1.
OBJECTIVES/HYPOTHESIS: To this date the effect of the time taken to complete an evaluation on the psychometric properties of the instrument has not been reported. The goal of our study was to assess the effect of time taken to complete an evaluation on its validity.
Cross-sectional validation study.
The global and checklist parts of tonsillectomy, mastoidectomy, rigid bronchoscopy, and endoscopic sinus surgery were used in the operating room by the otolaryngology faculty to evaluate the surgical skills of the residents. We categorized evaluations into two groups depending on the time taken to complete an evaluation (group A ≤ 6 days, group B >6 days). Construct validity was calculated for both groups by comparing the mean global and checklist scores of the residents across advancing postgraduate year levels.
A total of 468 evaluations, consisting of global and checklist parts, were completed for 29 residents by 32 evaluators. Mean number of days taken to complete an evaluation was 7.7 days. For all the evaluations completed within a 6-day time period, the construct validity was significant for both global and checklist parts of the four instruments. In cases of the evaluations completed after 6 days, the construct validity was significant for the tonsillectomy instrument only.
Our results indicate that the time taken to complete an evaluation has a significant effect on the construct validity of the objective instrument. In the future, efforts should be focused on faculty development to ensure timely completion of the evaluation for a more valid assessment process.
目的/假设：迄今为止，完成评估所花费的时间对仪器的心理测量特性的影响尚未报道。我们的研究目的是评估完成评估所需时间对其有效性的影响。
横断面验证研究。
耳鼻喉科教员在手术室使用扁桃体切除术、乳突切除术、硬性支气管镜检查和鼻内镜鼻窦手术的全球和检查表部分来评估住院医师的手术技能。我们根据完成评估所花费的时间将评估分为两组（A 组≤6 天，B 组>6 天）。通过比较住院医师在研究生水平上的全球和检查表分数的平均值，计算两组的构念效度。
共有 32 名评估员为 29 名住院医师完成了 468 次评估，包括全球和检查表部分。完成评估所花费的平均天数为 7.7 天。对于在 6 天内完成的所有评估，四个仪器的全球和检查表部分的构念效度均具有统计学意义。在 6 天后完成的评估中，仅扁桃体切除术仪器的构念效度具有统计学意义。
我们的结果表明，完成评估所花费的时间对客观仪器的构念效度有显著影响。未来，应集中精力进行教员发展，以确保及时完成评估，从而进行更有效的评估过程。
