Enhancing Traditional ITS Architectures with Large Language Models for Generating Motivational Feedback

Large Language Models (LLMs) have emerged as powerful computational tools with significant potential across various fields, particularly education. Previous research demonstrated that LLMs can be effectively integrated into the Tutoring Model of Intelligent Tutoring Systems (ITS) to generate motivational feedback within learning environments. To further explore this area, the authors define an ITS architecture that integrates an LLM (referred to as the ITS-LLM architecture) and validates this architecture by implementing a prototype using Llama 3.2 model. Moreover, the prototype aims to evaluate the impact of LLM-generated motivational feedback on student motivation and their learning achievements. This work outlines the ITS-LLM architecture, the prototype, and the evaluation methodology, which includes a case study involving two groups of students: one receiving LLM-generated feedback and the other receiving feedback from human experts (expert-generated feedback). The application of Situational Motivation Scale (SIMS) and Classical Test Theory (CTT) demonstrated that both the aforementioned types of feedback positively influence student motivation and learning achievements. Notably, LLM-generated feedback excelled in providing personalized feedback, while expert-generated feedback highlighted the importance of human empathy. These results underscore the effectiveness of LLMs in engaging students and enhancing their academic performance.