Generative AI-Based Virtual Assistant Using Retrieval-Augmented Generation: An evaluation study for bachelor projects (BNAIC/BeNeLearn 2024)
This paper develops a Retrieval-Augmented Generation virtual assistant to help Maastricht University students navigate project regulations, improving accuracy and reliability in specialized educational contexts using Large Language Models.
Abstract: Large Language Models have been increasingly employed in the creation of Virtual Assistants due to their ability to generate human-like text and handle complex inquiries. While these models hold great promise, challenges such as hallucinations, missing information, and the difficulty of providing accurate and context-specific responses persist, particularly when applied to highly specialized content domains. In this paper, we focus on addressing these challenges by developing a virtual assistant designed to support students at Maastricht University in navigating project-specific regulations. We propose a virtual assistant based on a Retrieval-Augmented Generation system that enhances the accuracy and reliability of responses by integrating up-to-date, domain specific knowledge. Through a robust evaluation framework and real-life testing, we demonstrate that our virtual assistant can effectively meet the needs of students while addressing the inherent challenges of applying Large Language Models to a specialized educational context. This work contributes to the ongoing discourse on improving LLM-based systems for specific applications and highlights areas for further research.
https://youtu.be/jXC7-cPD4Z4?si=fgBvfTeth9PCYYoC
Authors: Dumitru Versebeniuc, Martijn Elands, Sara Falahatkar, Chiara Magrone, Mohammad Falah, Martijn Bousse and Aki Harma
Conference: BNAIC/BeNeLearn 2024, Utrecht.