A Multimodal AI Tutor: Conversational Guidance with Dynamic Visualizations
MSc Thesis of Tommy Tran, defended on Thursday, 9th of October, 2025 at DACS, Maastricht University, The Netherlands
Abstract: One-on-one tutoring is highly effective, but most schools and universities cannot provide it at scale. Conventional Intelligent Tutoring Systems attempted to fill this gap. But their reliance on handcrafted rules and scripted dialogues made them costly and challenging to scale. Advances in large language models, including multimodal LLMs that process text and images, offer new opportunities for scalable tutoring. This thesis presents a multimodal AI tutor. The system integrates GPT-4.1 with a chat interface and interactive whiteboard, allowing students to write formulas, draw graphs, and annotate visuals. At the same time, the AI interprets and responds with generated text and graphics. A multi-agent architecture separates dialogue, mathematical validation, and visual generation to improve performance and reliability. Prompt engineering aligned the tutor with pedagogy, using scaffolding and Socratic questioning to guide students rather than supply direct answers. In a user study where students, staff, and professionals solved a statistics task, the tutor demonstrated the ability to provide context-aware feedback, valuable visual interactions, and pedagogical alignment. Challenges such as hallucinations and sycophancy persist, but findings suggest that multimodal LLMs can offer the benefits of one-on-one tutoring and should be explored as a complement to classroom teaching.
Cite: Tommy Tran, A Multimodal AI Tutor: Conversational Guidance with Dynamic Visualizations, MSc Thesis, DACS/FSE, Maastricht University, October 2025.