Does weekly direct observation and formal feedback improve intern patient care skills development? A randomized controlled trial.

BACKGROUND AND OBJECTIVES

Direct observation (DO) is considered to be an effective way to evaluate patient care. This study assesses if weekly direct observation with formal feedback (DO-FF) increases (1) clinical skills and (2) comfort with patient care skills (CWPCS) during the first 12 weeks of internship.

METHODS

A single-blinded, stratified, randomized controlled trial (RCT) with allocation concealment incorporating interns at a community hospital family medicine residency program was performed. Interns (n=14) were stratified by the predicted number of calls in a 2:1 ratio of intervention: control group. The intervention group received DO-FF four times/month on inpatient history and physicals (H&Ps) by a family medicine senior resident or fellow. To assess skills, all interns were videotaped doing H&Ps at the beginning, middle, and end of the study. These were scored by two independent, blinded physicians using a validated tool. For self-assessment, all interns took a patient care comfort survey (PCCS) at baseline, 6 weeks, and 12 weeks. Analysis was done via intention to treat.

RESULTS

At 6 weeks, interns who received weekly DO-FF had greater CWPCS on 32 of 35 questions. Eight were significantly different: (1) discussing end of life issues, (2) chest X ray interpretation, management of (3) congestive heart failure, (4) chronic obstructive pulmonary disease, (5) diabetes mellitus, (6) diabetic ketoacidosis, (7) stroke, and (8) venous thromboembolism/pulmonary embolism. In all but three questions, differences resolved by 12 weeks. Video data were not significantly different between groups.

CONCLUSIONS

In certain aspects of patient care, DO-FF improved intern CWPCS at 6 weeks. There was no difference in the video assessments of clinical skills between the treatment groups. These results need to be further explored. However, this study showed that a well-designed RCT is feasible for educational research questions.