Metzner Olivia, Wang Yindong, Symes Wendy, Huang Yizhen, Keller Lena, de Melo Gerard, Lazarides Rebecca
Department of Educational Sciences, University of Potsdam, Potsdam, Germany.
Chair of Artificial Intelligence and Intelligent Systems, Hasso-Plattner-Institute/University of Potsdam, Potsdam, Germany.
Br J Educ Psychol. 2025 Sep;95 Suppl 1(Suppl 1):S73-S97. doi: 10.1111/bjep.12779. Epub 2025 Apr 28.
Recent studies have examined the relation between teacher motivation, motivational messages and student learning but are limited to an achievement-related context, primarily using survey data. Moreover, our understanding of the relation between various teacher characteristics, such as teacher self-efficacy (TSE), and their motivational message use remains limited.
Our study tested whether teacher speech can be classified into self-determination (SDT)-based motivational messages and reliably assessed with a large language model (LLM). Additionally, we analysed the relation between pre-service TSE and their motivational message use.
For our first aim, we used human-rater annotations from 119 pre-service teachers' classroom recordings. For our second aim, we used data from 103 pre-service teachers (52.69% female; M = 22.98, SD = 3.26, Min = 19, Max = 34) who participated in a survey and were video-recorded while teaching.
First, we manually classified pre-service teachers' motivational messages based on transcripts and used human-rater annotations to fine-tune an LLM. Second, we analysed the relation between pre-service TSE and motivational message use.
The fine-tuned LLM demonstrated promising performance in assessing SDT-based motivational messages but needs further refining to assess thwarting messages. The analysis with human annotation showed that pre-service TSE for classroom management positively affected the frequency of relatedness-supportive messages. Pre-service TSE for student engagement increased the likelihood of never using a competence- or relatedness-thwarting message. Pre-service TSE for instructional strategies reduced the frequency of autonomy-supportive messages. LLM-based analyses showed slightly different results but did not contradict human annotation-based analyses.
近期研究探讨了教师动机、激励信息与学生学习之间的关系，但仅限于与成就相关的背景，主要使用调查数据。此外，我们对各种教师特征（如教师自我效能感）与他们使用激励信息之间的关系的理解仍然有限。
我们的研究测试了教师言语是否可以归类为基于自我决定理论（SDT）的激励信息，并能否通过大语言模型（LLM）进行可靠评估。此外，我们分析了职前教师自我效能感与他们使用激励信息之间的关系。
对于我们的第一个目标，我们使用了119名职前教师课堂录音的人工评分注释。对于我们的第二个目标，我们使用了103名职前教师（52.69%为女性；M = 22.98，SD = 3.26，最小值 = 19，最大值 = 34）的数据，他们参与了一项调查，并在教学时进行了视频录制。
首先，我们根据文字记录对手职前教师的激励信息进行人工分类，并使用人工评分注释对大语言模型进行微调。其次，我们分析了职前教师自我效能感与激励信息使用之间的关系。
经过微调的大语言模型在评估基于自我决定理论的激励信息方面表现出了良好的性能，但需要进一步完善以评估阻碍性信息。人工注释分析表明，职前教师课堂管理的自我效能感对相关性支持信息的频率有积极影响。职前教师学生参与度的自我效能感增加了从不使用能力或相关性阻碍信息的可能性。职前教师教学策略的自我效能感降低了自主性支持信息的频率。基于大语言模型的分析结果略有不同，但与基于人工注释的分析结果并不矛盾。
