Treadwell Ina
Skills Laboratory, Faculty of Health Science, University of Pretoria, Pretoria, South Africa.
Med Educ. 2006 Sep;40(9):855-61. doi: 10.1111/j.1365-2929.2006.02543.x.
The administration of an objective structured clinical examination (OSCE) using paper checklists presents problems such as illegible handwriting, missing student names and/or numbers and lost checklists. Calculating and entering results is not only time-consuming, but is subject to human errors, and feedback to students is rarely available. To rectify these problems, personal digital assistants (PDAs) and HaPerT software were acquired to replace paper checklists and provide automated results and feedback. This study sought to determine the usability of the PDA assessment system. The usability of the PDA system was evaluated according to effectiveness, efficiency and user satisfaction. Effectiveness was judged by comparing the results of an OSCE conducted in 2003 (paper-based method) and repeated in 2004 and 2005 (PDA method). Efficiency was determined by calculating the amount of time required to organise the logistics for 2 consecutive PDA OSCEs and deliver results and grades, compared with the time required for the paper-based OSCE. User satisfaction was established by using questionnaires to obtain feedback on the assessors' experiences during their first assessments. An independent groups t-test used to compare the means of scores achieved by students in the PDA and paper-based OSCEs, respectively, showed that the difference in effectiveness was not significant. In terms of efficiency, 77% less time was used for the PDA OSCE in 2004 and 93% less in 2005. Assessor feedback on PDA assessment was overwhelmingly positive. Assessment by PDA was found to be just as effective as and more efficient than paper-based assessment in practical examinations, and was highly rated by assessors.
使用纸质检查表进行客观结构化临床考试（OSCE）存在诸多问题，如字迹难以辨认、学生姓名和/或学号缺失以及检查表丢失等。计算和录入结果不仅耗时，还容易出现人为错误，而且很少能向学生提供反馈。为纠正这些问题，购置了个人数字助理（PDA）和HaPerT软件来取代纸质检查表，并提供自动生成的结果和反馈。本研究旨在确定PDA评估系统的可用性。根据有效性、效率和用户满意度对PDA系统的可用性进行了评估。通过比较2003年采用纸质方法进行的OSCE结果与2004年和2005年重复进行的（采用PDA方法）OSCE结果来判断有效性。通过计算连续两次PDA OSCE组织后勤工作、给出结果和成绩所需的时间，并与纸质OSCE所需时间进行比较来确定效率。通过问卷调查获取评估人员首次评估期间的体验反馈来确定用户满意度。分别用于比较PDA和纸质OSCE中学生成绩均值的独立样本t检验表明，有效性方面的差异不显著。在效率方面，2004年PDA OSCE所用时间减少了77%，2005年减少了93%。评估人员对PDA评估的反馈总体上是积极的。发现在实践考试中，PDA评估与纸质评估同样有效且效率更高，并且受到评估人员的高度评价。
