CheatGPT: how will teachers respond?

Now we understand a bit more about how ChatGPT works, it’s time to have a think about what sort of immediate impacts might be felt by society as a result of its introduction, and how it could change things further into the future. Earlier, we mentioned that some schools in America have already taken a swift, hardline approach to students using ChatGPT as a fast way to finish assignments, banning the chatbot from computers hooked up to the school’s network. Vitomir Kovanovic is a senior lecturer at the Centre for Change and Complexity in Learning at the University of South Australia whose background is in Computer Science. He specialises in learning analytics, and told James about how large language models could change education for the worse, but also for the better…

Vitomir - We had a conversation with teachers in South Australia about what are the ways you can use this? And there are several really potentially beneficial ways using it to very quickly generate a set of exam questions that haven't been used in the past, for example. Previously, educators used question banks. With a colleague, we were just testing the system and we used it to generate the syllabus of the course. It actually produced a very impressive syllabus. And then we said, how about you put a little bit more practical work in weeks five to seven or whatever, and then the system generates another one. That's how you have to use it. This kind of system will require a lot of skills to use. It's literally almost as if you hired a great composer and now you want him to compose something for you. You really need to be able to articulate what you want and be able, when it comes back with something, to say, "ah, this is not what I actually wanted. I wanted something slightly more dynamic." "I wanted more darker tones." You need to understand and be able to communicate with the machine to achieve what you really want.

James - It's interesting you framing it in the ways it can be used to the benefit of education. I completely agree with those points, but I wonder if we could just rewind a bit and think again about how, in the hands of the students, especially in the short term, playing around with the technology myself I can see how it would take a lot of the heavy lifting out of a task of an essay you've been set for homework. And especially if we continue to see improvements and students learn how to better use the software themselves, presumably this is a point of some concern?

Vitomir - Right away it'll really completely invalidate all the assessments we do because, let's be honest, a big part of the assessment is writing essays, long written responses, and so on. But you can literally say, write me a response to this question pretending that you're a year eight student and it'll simulate the stylistic complexity that that would be expected of a year eight or year nine. I'm pretty sure students are starting to use it. A bigger question is, "is this a good way to assess their learning?"

James - Is it possible simply to just police this properly? Can we not just ban the websites on school networks to stop students being able to use it?

Vitomir - Well on school networks, yes. Things like that are very easy to do. But the problem is these systems will become more and more common and, in a sense, why would you? The only reason to do it is to protect the existing assessment models. And we already know they're really not fit for purpose, so we want to change them.

James - I've heard ChatGPT be described before as like a calculator, but for essay writing and idea generation. Calculators, obviously when they became inexpensive and widely available, didn't make maths a redundant subject. Is that a comparison or an analogy you like? Or is ChatGPT even more powerful than, say,a calculator? Will it usher in even more dramatic change?

Vitomir - The comparison makes sense, but it's much more powerful. Grammar check would be something like a calculator; something that does something very small, constrained, and it does help, right? Writing a good essay with spellcheck or without spellcheck, it's not the same. But this is much, much more than that. It's almost like having a professional mathematician sitting next to you. What's interesting is, looking at the responses, people in the 60's, when calculators massively became available, that was the same discussion. Should we allow calculators? Shouldn't we allow them? But solving a big complex mathematical problem requires you to use a calculator 50 times, but how do you assemble the steps? Really critical thinking of solving mathematical problems, you still had to do it yourself. This system is far more powerful than that. I mean, you can still simulate at least some bit of this critical thinking, but if you want to really produce a good essay or a good written response, you have to still intervene there. You're not just writing now, you're being given by the computer a written response and now, as a student, you need to evaluate, is it good enough for my task? You need to go back to the computer and give it more instructions, how do I fix it? And so on. So it'll be far more back and forth.

James - The landscape just feels like it's changing so quickly. What sort of thing are we talking about when we say 'change the assessment?' You mentioned the archaic way of doing it is by just saying "no technology," but it seems like teachers will have a responsibility to integrate this. People will be using this in the workplace before too long.

Vitomir - It's still really open. We first need to see how people are going to be using this. How would somebody writing a script use this? What's the potential of this? Because this is a very, very fuzzy, different technology from the others. Typically, when you develop a technology, you know what the possibilities are? We are not even sure what this technology can do. Let's say you're given an essay on pros and cons of abortion laws or any complex social issue - really what you're testing there is for critical thinking, their ability to critically summarise different ideas, integrate them, compare them, see where the difference really is. And you'll still have to do that. So the focus should be on that. It'll shift a little bit from being focused on writing perfectly, because machines can do that now. We'll focus on your logic. What are you really writing there? I think in the future we'll see assessments becoming more complicated and more demanding.