Department of Developmental and Educational Psychology, Faculty of Psychology, University of Valencia, Blasco Ibáñez Av. no. 21, Valencia, 46010, Spain.
Neurophysiotherapy Teaching Innovation Group, Department of Physiotherapy, Faculty of Physiotherapy, University of Valencia, Gascó Oliag Street no. 5, Valencia, 46010, Spain.
BMC Psychol. 2024 Mar 14;12(1):147. doi: 10.1186/s40359-024-01643-7.
The development of procedural skills is essential in health sciences education. Rubrics can be useful for learning and assessing these skills. To this end, a set of rubrics were developed in case of neurophysiotherapy maneuvers for undergraduates. Although students found the rubrics to be valid and useful in previous courses, the analysis of the practical exam results showed the need to change them in order to improve their validity and reliability, especially when used for summative purposes. After reviewing the rubrics, this paper analyzes their validity and reliability for promoting the learning of neurophysiotherapy maneuvers and assessing the acquisition of the procedural skills they involve.
In this cross-sectional and psychometric study, six experts and 142 undergraduate students of a neurophysiotherapy subject from a Spanish university participated. The rubrics' validity (content and structural) and reliability (inter-rater and internal consistency) were analyzed. The students' scores in the subject practical exam derived from the application of the rubrics, as well as the rubrics' criteria difficulty and discrimination indices were also determined.
The rubrics´ content validity was found to be adequate (Content Validity Index > 0.90). These showed a unidimensional structure, and an acceptable internal consistency (α = 0.71) and inter-rater reliability (Fleiss' ƙ=0.44, ICC = 0.94). The scores of the subject practical exam practically covered the entire range of possible theoretical scores, showing all the criterion medium-low to medium difficulty indices - except for the one related to the physical therapist position-. All the criterion exhibited adequate discrimination indices (rpbis > 0.39), as did the rubric as a whole (Ferguson's δ = 0.86). Students highlighted the rubrics´ usefulness for learning the maneuvers, as well as their validity and reliability for formative and summative assessment.
The changed rubrics constitute a valid and reliable instrument for evaluating the execution quality of neurophysiotherapy maneuvers from a summative evaluation viewpoint. This study facilitates the development of rubrics aimed at promoting different practical skills in health-science education.
程序技能的发展在健康科学教育中至关重要。评分标准可用于学习和评估这些技能。为此，针对大学生神经物理治疗手法，制定了一套评分标准。尽管学生们在前几门课程中发现这些评分标准是有效的且有用的，但实际考试结果的分析表明，需要对其进行修改，以提高其有效性和可靠性，尤其是在用于总结性评估时。在对评分标准进行审查后，本文分析了其有效性和可靠性，以促进神经物理治疗手法的学习，并评估所涉及的程序技能的掌握情况。
在这项横断面和心理测量研究中，西班牙一所大学的神经物理治疗科目有 6 名专家和 142 名本科生参与。分析了评分标准的有效性（内容和结构）和可靠性（评分者间和内部一致性）。学生在该科目的实际考试中的分数来自于评分标准的应用，还确定了评分标准的标准难度和区分度指数。
评分标准的内容有效性被认为是充分的（内容效度指数>0.90）。这些评分标准具有一维结构，内部一致性（α=0.71）和评分者间可靠性（Fleiss' ƙ=0.44，ICC=0.94）可接受。该科目的实际考试成绩几乎涵盖了所有可能的理论分数范围，除了与物理治疗师位置相关的那个指标外，所有标准都显示出中低到中等难度指数-。所有标准都表现出足够的区分度指数（rpbis>0.39），整个评分标准也是如此（Ferguson 的 δ=0.86）。学生们强调了评分标准对学习手法的有用性，以及其对形成性和总结性评估的有效性和可靠性。
修改后的评分标准构成了一种有效的、可靠的工具，可从总结性评估的角度评估神经物理治疗手法的执行质量。本研究促进了旨在促进健康科学教育中不同实践技能发展的评分标准的制定。
