Design of a user-centered voluntary patient safety reporting system: understanding the time and response variances by retrospective think-aloud protocols.

Usability is critical to the success of computerized systems, and yet it has received little attention in the field of voluntary patient safety reporting. Failures in this regard may largely account for the issues of low user acceptance and low-quality data that currently confront the system. In this study, we examined the three usability aspects of efficiency, effectiveness and user attitudes on a typical safety reporting system. The system usability was evaluated using the retrospective think-aloud testing method, which measures execution time and response consistency with think-aloud protocols. Ten end-users were recruited for the test. The descriptive statistics on users' time and response variances unveiled system features that influenced the system's reporting efficiency and effectiveness. The think-aloud protocols that reflected users' attitudes helped identify nine categories of usability problems associated with the response variances and system features. In the end, the observed semantic ambiguity, terminology complexity and carry-over effect are noted as challenges and opportunities for further usability improvements.