BACKGROUND: Motor skills assessment is an important part of validating surgical competency. The need to test surgical skills competency has gained acceptance; however, assessment methods have not yet been defined or validated. The purpose of the present study was to evaluate the reliability and validity of four testing measures for the integrated assessment of orthopaedic surgery residents with regard to their competence in performing carpal tunnel release. METHODS: Twenty-eight orthopaedic residents representing six levels of surgical training were tested for competence in performing carpal tunnel release on cadaver specimens. Four measures were used to assess competency. First, a web-based knowledge test of surgical anatomy, surgical indications, surgical steps, operative report dictation, and surgical complications was administered. Second, residents participated in an Objective Structured Assessment of Technical Skills; each resident performed surgery on a cadaver specimen. All residents were evaluated independently by two board-certified orthopaedic surgeons with a subspecialty certificate in hand surgery with use of a detailed checklist score, a global rating scale, and a pass/fail assessment. The time for completion of the surgery was also recorded. Each assessment tool was correlated with the others as well as with the resident's level of training. RESULTS: Significant differences were found between year of training and knowledge test scores (F = 7.913, p < 0.001), year of training and detailed checklist scores (F = 5.734, p = 0.002), year of training and global rating scale (F = 2.835, p = 0.040), and year of training and percentage pass rate (F = 26.3, p < 0.001). No significant differences were found between year of training and time to completion of the carpal tunnel release (F = 2.482, p < 0.063). CONCLUSIONS: The results of the present study suggest that both knowledge and cadaver testing discriminate between novice and accomplished residents. However, although failure of the knowledge test can predict failure on technical skills testing, the presence of knowledge does not necessarily ensure successful performance of technical skills, as cognitive testing and technical skills testing are separate domains.