“You can ask a chatbot things you might not dare to ask in a lecture”

Jan Vermant, how has generative AI changed teaching since the boom of ChatGPT?
As lecturers, we face the challenge of ensuring, through exams and written submissions, that our students’ abilities take centre stage, rather than the abilities of a language model. This means we need to think carefully about how to design exam formats that reflect students’ individual skills and creative approaches.

How have you responded yourself?
I do more oral exams now. Discussions with students and feedback from teaching assistants have become more important. This makes examining students a more intensive and laborious task for lecturers.

In which areas have you seen the benefits of language models?
In programming. For example, in one of my courses on fluid dynamics, the students each have to program a small application. In the past, they often lost time on this task due to errors in the code. Thanks to language models, they no longer make these mistakes, and we can deal with the actual content of the course, physics, more quickly. However, we also need to spend time on critically discussing the output of AI models.

How does ETH plan to equip its students for this kind of critical reflection?
The use of generative AI will be part of the skills we need to teach, and there are excellent examples of how this has already been implemented – including in fields other than computer science. In civil engineering, for example, AI skills are gradually built up in a highly structured manner as part of a course on digital engineering. In that course, students work with language models in order to understand program code better, to complete code, to identify errors and to provide documentation for the code. Likewise, a biology course uses an AI tutor that applies teaching materials and assists the students with targeted questions in order to deepen their knowledge. However, there are also other courses that use generative AI and that we promote or have promoted via our funding for innovations in teaching.

In which role do you think generative AI offers the greatest potential for teaching?

As a tutor – in the sense that we can use generative AI to automatically provide personalised feedback to students. As part of the Ethel project, ETH is testing course-specific chatbots that support learning or can correct exercise submissions based on the respective course materials. The advantage is that they do this as often as you want, at any time of the day or night. You can also ask a chatbot questions that you might not dare to ask in a large auditorium. In the long term, I think there’s potential in this low-threshold interaction and in the immediate, individual feedback that can be provided, but we’re still in the early stages.

ETH Professor Torbjørn Netland recently carried out experiments for a research project using AI avatars in educational videos. Can we soon replace lecturers with AI avatars?
If you put it like that, no. A university education requires interaction with humans. Knowledge must be contextualised and continually differentiated if we are to achieve the standards for which ETH is famous. However, Netland doesn’t want to replace lecturers with avatars, particularly not in a lecture theatre. He investigated whether videos generated with the help of generative AI offered advantages or disadvantages compared with other videos (see box). It’s clear that the straightforward transfer of knowledge can be restructured with the help of generative AI. But I’m convinced that language models will always just be a teaching aid and will not replace human interaction. After all, our objective is to challenge students to think one step ahead and dig deeper. That’s difficult for a language model to do.

How is generative AI changing curricula?
Of course, there needs to be space for methods and background knowledge: How do I write prompts in academia? What kind of mistakes do the models make? In which areas do they give a distorted picture? In principle, I think lecturers will probably need to devote more time to practising and encouraging critical thinking with students in future. This is time that the lecturers might gain by making use of AI in the straightforward transfer of knowledge.

What do you think are the limits of language models in teaching?
In my field, what I’ve seen is that the majority are not necessarily doing the same as what the best are doing. However, the language models often fail to reflect this pioneering knowledge and instead offer an “average”, so to speak. I’ve asked language models about my field – and, so far, the answers haven’t been all that intelligent.

What are the biggest challenges for the future?
At present, we’re reliant on the big players, even though we have skills they lack. I wonder how we can leverage our own strengths with a view to building up teaching opportunities of our own.