Is the Accreditation Council for Graduate Medical Education (ACGME) Resident/Fellow survey, a valid tool to assess general surgery residency programs compliance with work hours regulations?

The Accreditation Council for Graduate Medical Education (ACGME) uses the resident/fellow survey to assess residency programs compliance with ACGME work hours regulations. Survey results can have significant consequences for residency programs including ACGME letters of warning, shortened program accreditation cycle, immediate full program and institutional site visits, or administrative withdrawal of a program's accreditation. Survey validity was assessed by direct query of general surgery residents who answer the survey each year. A multiple-choice survey was created to assess all US general surgery residents' interpretation and understanding of the ACGME survey. The survey was distributed to all surgery residency program directors in the US in 2009. Responses were compiled via an online survey program. Statistical analysis was performed in aggregate and between junior and senior residents. Nine hundred sixty-five (13.2%) general surgical residents responded with 961 (99.6%) completing all questions. All responding residents had taken the ACGME survey at least once with 634 (66%) having taken it more than once. Nineteen percent of residents had difficulty understanding the questions with senior residents (23%) reporting difficulty more than junior residents (14%), p < 0.001. Thirty-five percent of residents had discussed the survey with their faculty or program director prior to taking it, while 17% were instructed on how to answer the survey. One hundred thirty-three residents (14%) admitted to not answering the questions truthfully while 352 (37%) of residents felt that the survey did not provide an accurate evaluation of their work hours in residency training. An evaluation tool in which 1 in 7 residents admit to answering the questions falsely and 1 in 5 residents had difficulty interpreting the questions may not be a valid method to evaluate compliance with work hours regulations. Evaluation of work hours regulations compliance should be based on actual work hours data rather than an anonymous survey.