Mayo Clinic, 200 First Street, SW, Rochester, MN, 55902, USA.
BMC Med Educ. 2020 Mar 4;20(1):64. doi: 10.1186/s12909-020-1968-1.
Learners may subconsciously change their behavior once they know they are being observed, and this Hawthorne effect should be considered when designing assessments of learner behavior. While there is a growing body of literature to suggest direct observation is the ideal standard for formative assessment, the best method to directly observe learners is unknown. We explored scheduled and unscheduled methods of direct observation among internal medicine residents in the outpatient continuity clinic to advance the understanding of both observation methods.
We conducted a thematic analysis of faculty and internal medicine residents in an outpatient clinic setting. A semi-structured interview guide for focus group sessions was created. Focus groups were used to explore the internal medicine resident and core teaching faculty perceptions of the scheduled and unscheduled direct observation methods in the outpatient clinc. An experienced qualitative research interviewer external to the internal medicine residency was moderating the sessions. Eight peer focus groups were held. Abstraction of themes from focus group transcripts identified resident and faculty perceptions of the different observation methods.
Focus groups had 14 resident participants and 14 faculty participants. Unscheduled observations were felt to be more authentic than scheduled observations since residents perceived their behavior to be unmodified. Unscheduled observations allowed for increased numbers of observations per resident, which permitted more frequent formative assessments. Residents and faculty preferred remote video observation compared to in-room observation. Participants found direct observation a useful learning tool for high-yield, specific feedback.
Unscheduled remote direct observation captures authentic clinical encounters while minimizing learner behavior modification. An unscheduled observation approach results in more frequent formative assessment and therefore in more instances of valuable feedback compared to scheduled observations. These findings can help guide the best practice approaches to direct clinical observation in order to enhance residents learning and experience.
一旦学习者知道自己正在被观察，他们可能会下意识地改变自己的行为，因此在设计对学习者行为的评估时应考虑到这种霍桑效应。虽然越来越多的文献表明直接观察是形成性评估的理想标准，但直接观察学习者的最佳方法尚不清楚。我们在门诊连续性诊所的内科住院医师中探索了直接观察的计划和非计划方法，以深入了解这两种观察方法。
我们在内科门诊环境中对教师和内科住院医师进行了主题分析。为焦点小组会议创建了半结构化访谈指南。焦点小组用于探讨内科住院医师和核心教学教师对门诊诊所中计划和非计划直接观察方法的看法。一位经验丰富的外部内科住院医师的定性研究访谈员主持了会议。共举行了 8 次同行焦点小组会议。从焦点小组记录中提取主题，以确定住院医师和教师对不同观察方法的看法。
焦点小组有 14 名住院医师和 14 名教师参加。与计划观察相比，非计划观察被认为更真实，因为住院医师认为他们的行为没有被修改。非计划观察允许每个住院医师进行更多次数的观察，从而可以更频繁地进行形成性评估。住院医师和教师更喜欢远程视频观察而不是室内观察。参与者认为直接观察是一种有用的学习工具，可以提供高收益、具体的反馈。
非计划远程直接观察可以在最小化学习者行为改变的情况下捕捉真实的临床体验。与计划观察相比，非计划观察方法可进行更频繁的形成性评估，因此可提供更多有价值反馈的实例。这些发现可以帮助指导直接临床观察的最佳实践方法，以提高住院医师的学习和体验。
