Using Rasch Analysis for Validation of Knowledge and Perception of Orthopedic Workplace-Based Assessment among Postgraduate Orthopedic Trainees' Questionnaire.

INTRODUCTION/BACKGROUND: Workplace-based assessment (WPBA) was formally introduced into orthopedic postgraduate training in Malaysia as of December 2022. Understanding users' knowledge and perception (KAP) of WPBA is essential for ensuring its effective implementation. A systematic review is needed to evaluate whether WPBA is meeting educational expectations.

OBJECTIVE

This study aimed to validate a questionnaire designed to assess the knowledge and perception of WPBA among postgraduate orthopedic trainees.

MATERIALS AND METHODS

A pilot study was conducted from Aug 1, 2022 to May 31, 2023 involving a 28-item questionnaire that had undergone thorough face and content validation. The questionnaire was distributed to orthopedic postgraduate trainees at the Department of Orthopaedics, Traumatology and Rehabilitation, Kulliyyah of Medicine, International Islamic University Malaysia (IIUM). Rasch analysis was performed to evaluate the psychometric properties of the instrument.

RESULTS

The Rasch analysis revealed a person reliability index of 0.93 and a separation index of 3.62, while the item reliability index was 0.99 with a separation index of 8.32. The mean infit and outfit mean square (MNSQ) values were 0.99 and 0.96 respectively, with Z-standard (Z-STD) values of 0.24 and -0.35. Several misfitting items fell outside the acceptable range (0.6-1.4) and were subsequently removed from the questionnaire.

CONCLUSIONS

The validation and reliability testing confirmed that the orthopedic WPBA KAP questionnaire recently developed is both reliable and valid for assessing postgraduate trainees' understanding of WPBA.