Miami University, Oxford, OH 45056, USA.
Behav Res Methods. 2013 Sep;45(3):623-36. doi: 10.3758/s13428-013-0352-z.
The goal of intelligent tutoring systems (ITS) that interact in natural language is to emulate the benefits that a well-trained human tutor provides to students, by interpreting student answers and appropriately responding in order to encourage elaboration. BRCA Gist is an ITS developed using AutoTutor Lite, a Web-based version of AutoTutor. Fuzzy-trace theory theoretically motivated the development of BRCA Gist, which engages people in tutorial dialogues to teach them about genetic breast cancer risk. We describe an empirical method to create tutorial dialogues and fine-tune the calibration of BRCA Gist's semantic processing engine without a team of computer scientists. We created five interactive dialogues centered on pedagogic questions such as "What should someone do if she receives a positive result for genetic risk of breast cancer?" This method involved an iterative refinement process of repeated testing with different texts and successively making adjustments to the tutor's expectations and settings in order to improve performance. The goal of this method was to enable BRCA Gist to interpret and respond to answers in a manner that best facilitated learning. We developed a method to analyze the efficacy of the tutor's dialogues. We found that BRCA Gist's assessment of participants' answers was highly correlated with the quality of the answers found by trained human judges using a reliable rubric. The dialogue quality between users and BRCA Gist predicted performance on a breast cancer risk knowledge test completed after exposure to the tutor. The appropriateness of BRCA Gist's feedback also predicted the quality of answers and breast cancer risk knowledge test scores.
智能辅导系统（ITS）以自然语言进行交互的目标是模拟经过良好培训的人类导师为学生提供的好处，通过解释学生的答案并做出适当的回应，以鼓励学生详细阐述。BRCA Gist 是使用 AutoTutor Lite 开发的 ITS，AutoTutor Lite 是 AutoTutor 的基于 Web 的版本。模糊痕迹理论从理论上激发了 BRCA Gist 的发展，它通过与人们进行辅导对话来教导他们有关乳腺癌风险的遗传知识。我们描述了一种创建辅导对话并微调 BRCA Gist 的语义处理引擎校准的经验方法，而无需计算机科学家团队。我们创建了五个以教学问题为中心的互动对话，例如“如果某人收到乳腺癌遗传风险的阳性结果，她应该怎么做？” 这种方法涉及一个迭代细化过程，使用不同的文本进行反复测试，并逐步调整导师的期望和设置，以提高性能。该方法的目的是使 BRCA Gist 能够以最有利于学习的方式解释和响应答案。我们开发了一种分析导师对话效果的方法。我们发现，BRCA Gist 对参与者答案的评估与经过培训的人类裁判使用可靠的评分标准找到的答案质量高度相关。用户与 BRCA Gist 之间的对话质量预测了在暴露于导师后完成的乳腺癌风险知识测试的表现。BRCA Gist 反馈的适当性也预测了答案的质量和乳腺癌风险知识测试的分数。
