CSEE&T 2024
Mon 29 July - Thu 1 August 2024 Würzburg, Germany
Wed 31 Jul 2024 13:00 - 13:20 at Room 1 - AI in Education and Training Chair(s): Andreas Bollin

This paper presents the use of Retrieval Augmented Generation (RAG) to improve the feedback generated by Large Language Models for programming tasks. For this purpose, corresponding lecture recordings were transcribed and made available to the Large Language Model GPT-4 as external knowledge source together with timestamps as metainformation by using RAG. The purpose of this is to prevent hallucinations and to enforce the use of the technical terms and phrases from the lecture. In an exercise platform developed to solve programming problems for an introductory programming lecture, students can request feedback on their solutions generated by GPT-4. For this task GPT-4 receives the students’ code solution, the compiler output, the result of unit tests and the relevant passages from the lecture notes available through the use of RAG as additional context. The feedback generated by GPT-4 should guide students to solve problems independently and link to the lecture content, using the time stamps of the transcript as meta-information. In this way, the corresponding lecture videos can be viewed immediately at the corresponding positions. For the evaluation, students worked with the tool in a workshop and decided for each feedback whether it should be extended by RAG or not. First results based on a questionnaire and the collected usage data show that the use of RAG can improve feedback generation and is preferred by students in some situations. Due to the slower speed of feedback generation, the benefits are situation dependent.

Wed 31 Jul

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

13:00 - 14:20
AI in Education and Training Research Track at Room 1
Chair(s): Andreas Bollin University of Klagenfurt, Austria
13:00
20m
Talk
Leveraging Lecture Content for Improved Feedback: Explorations with GPT-4 and Retrieval Augmented Generation
Research Track
Sven Jacobs Computer Science Education, University of Siegen, Steffen Jaschke
13:20
20m
Talk
A Survey Study on the State of the Art of Programming Exercise Generation using Large Language Models
Research Track
Eduard Frankford University of Innsbruck, Ingo Hoehn , Clemens Sauerwein University of Innsbruck, Ruth Breu University of Innsbruck
13:40
20m
Talk
Automated Programming Exercise Generation in the Era of Large Language Models
Research Track
Niklas Meissner Institute of Software Engineering, University of Stuttgart, Sandro Speth Institute of Software Engineering, University of Stuttgart, Steffen Becker University of Stuttgart
14:00
20m
Talk
Toward AI-facilitated Learning Cycle in Integration Course through Pair Programming with AI Agents
Research Track
Zhengyuan Wei City University of Hong Kong, Albert Ting-Leung Lee , Victor C. S. Lee Department of Electrical and Electronic Engineering, The University of Hong Kong, Wing-Kwong Chan City University of Hong Kong

Information for Participants
Wed 31 Jul 2024 13:00 - 14:20 at Room 1 - AI in Education and Training Chair(s): Andreas Bollin
Info for room Room 1:

Enter the building and take the main stairs or elevator to the top floor.