ChatGPT: The chatbot changing how we work

It's sensible not to take everything ChatGPT told you at face value, not least because the software admitted itself that you shouldn’t take everything it says as gospel. Oxford University’s Mike Wooldridge, who spoke to us when this story was breaking, is with us now to provide the human touch, and hopefully help us understand how all this is possible…

Mike - What's happened is that people have realised that scale matters in artificial intelligence. And what scale means for these systems is three things. Firstly, it means how big are your neural networks? Literally the larger your neural networks are, the more elements that they have. That matters. The amount of training data that you use to train your system - modern artificial intelligence absolutely relies on training data so that matters. And finally, the amount of compute effort that you are prepared to throw at training these programmes - that matters. And so there was this move that started around about five years ago that just said, "let's see how far we can take scale. Let's see how big our neural networks can get. Let's see how much data we can throw at these problems and let's see how much compute resource we are prepared to use." And the first system, the ancestor of ChatGPT was GPT-2, which I think appeared in 2018 or 19. Famously, it was supposedly so good that they were not prepared to release it to the public because this unprecedented power was too much for us to handle. But what happened with GPT-3, the successor system, is basically it was an order of magnitude bigger, an order of magnitude more data, an order of magnitude more compute power. And that's the way things are going. There's been a race for scale. That's what we're seeing. We're seeing the benefits of that.

Chris - I was just pressing James on how quickly it responded because normally you're used to your computer taking a while to load a game or something, and it's generating this output almost instantly as though it were just a human spouting a result back at you. What sort of computing grunt have they got on the back end of that to make that possible?

Mike - Okay, so you've got to distinguish two different things. Firstly, there's building, or training, the model; throwing the data at it to train it so that it learns how to be able to respond. That takes AI supercomputers running for months. Computationally it's one of the heaviest tasks that people are doing in computing. Now there's a big concern here about the amount of CO2 that's being generated while you're doing that. We believe that GPT-3, which is the technology that underpins ChatGPT, uses something like 24,000 GPUs - graphics processing units. And these are high performance AI computers running for a number of months in order to be able to churn through that data. So that's the training part. But once you've got that, you've got your neural network structures, actually using them, the runtime as we call it, what you were doing when you had your conversation, that's much cheaper but you're not going to do it on a desktop computer, you don't need anything like the scale. You don't need super computers to do that but you still need a lot more than a desktop computer. And the reason is those neural networks are very, very big. GPT-3 is 175 billion parameters. Basically, these are the 175 billion numbers that make up that neural network.

Chris - That's what I wanted to ask you about, because what has it actually learned? What is sitting in that machine that means when James asks it for its opinion on colors and it says, "well, I don't have one." How is it doing that?

Mike - There's a long answer and a short answer. The short answer is that we don't exactly know. The long answer is that basically what these things are doing is exactly the same as your smartphone does when it suggests a completion for you. So if you open up your smartphone and you start sending a text message to your partner saying, "I'm going to be...", it might suggest "late" or "in the pub." How is it doing that? Because it's looked at all the text messages that I've ever sent and it's seen that whenever I type "I'm going to be..." the likeliest next thing is going to be "late" or "in the pub." GPT systems are doing the same thing, but on a vastly larger scale. The training data for them is not the text messages you've sent, it's every bit of digital text that they could get their hands on when they wanted to train it. They download the entire internet and they train it using all of that text to try to make the prediction of what would be the next likeliest thing in the sentence.

Chris - The problem is, Mike, that the internet is full of rubbish. There's tons and tons of unreliable data out there. So how do you make sure that your system can sort wheat from chaff?

Mike - So you've put your finger on one of the big issues with this technology at the moment. There is so much data that it can't all be checked by humans before it's fed to the machine. Again, the details are sketchy on exactly how it happened in these public systems, but there will be some screening, probably automatic screening, looking for toxic content that will work to a certain extent. But it won't be reliable. It will get things wrong. It will allow through some things that really, ideally, we wouldn't allow. It will not be able to check the veracity of an awful lot of stuff that it's fed. What we're getting out of this is some kind of aggregate picture. It's like an average of what it's seen out there on the internet. But, to be honest, we need to do a ton more work to understand exactly what's going on there and exactly how we can deal with those issues. These are brand new tools that have landed on planet Earth and we've got a lot more work to do to understand them.

Chris - What can we expect to see this do next?

Mike - So the things that they're phenomenally good at are things to do with text. I urge you to try it: right? Go to to the BBC News website, cut and paste a story, and ask it to summarise it. In my experience, it usually does a very, very good job of coming out with the summary. Ask for a one paragraph summary, ask it to extract the top three bullet points from the news story, and it will do that. Take two news stories about the same thing and ask it to find what are the commonalities in the news story? What are the points of difference? It's in my experience also, the technology is very, very good at that. It's not perfect. You have to check it. It comes out with falsehoods, but it's very good. Where are you going to see it? You're going to see it in your email system. So instead of showing you every email, you're going to get the top three bullet points from your email. I think that would be quite a useful thing to be able to do.

One study published this week has demonstrated that ChatGPT, as well as being more than capable of generating homework indistinguishable from that of a real student, can even pen scientific content to a standard that allows it to go undetected as computer-made. Catherine Gao is a critical care physician at Northwestern University with a side hustle in machine learning research. She saw what ChatGPT was capable of and wondered how it would do at writing science abstracts - the summaries of the results of study papers published in journals. It blew her expectations out of the water and compelled her to systematically deduce just how indistinguishable it was from science written by real scientists…

Catherine - So one, we wondered if it would set off plagiarism detectors. The ChatGPT abstracts performed very well. They scored on average 100% original. So really not using any plagiarism in the traditional sense.

Chris - What that's telling you is that ChatGPT is not just going to some source online and grabbing wholesale that abstract and regurgitating it. It's generating content that is not in existence anywhere else.

Catherine - That's exactly right. It's really writing these abstracts from scratch. Another task that we looked at was whether or not it would score sort of high on different types of detectors. They exist online, these AI output detectors that you can run texts through. And so the real abstracts all scored very, very low. Most of them scored 0.02%, fake. Whereas the majority of the ChatGPT abstracts scored much higher with a median score of 99.98% fake.

Chris - So a machine can spot another machine's work.

Catherine - Yes, exactly. Using the machines to detect the machines. That's very right.

Chris - What about though, if you then thrust those results that it had generated in front of not a machine but people.

Catherine - Like I said, I was really impressed at how good it is. So we gave collections of 25 abstracts that were a mixture of real abstracts and generated abstracts to different team members. So these are all members who are within our biomedical sciences lab. So they're used to engaging with science. We said some of these are real, some of these are generated. Please give us a binary score of which one you think this is, and then you're also welcome to give us some notes to what made you think one way or the other. Even knowing that there are generated abstracts in this combined list, our human reviewers were only able to identify generated abstracts 68% of the time. And these were very, very suspicious, skeptical reviewers. They were so suspicious that they even thought 14% of the real abstracts were generated.

Chris - What about the quality of the content? Because you haven't said anything about that yet, whether or not when you looked at what the machine was saying, it was factually accurate?

Catherine - You know, at first I thought maybe it would provide some vague summary that was in the realm, but what we found is that in the generated abstracts ChatGPT actually came up with completely fabricated numbers for their results. Basically reporting full studies that just came out of the ether. What was really surprising to me was that it could hallucinate these numbers and present them in a way that seemed still factually sound enough that a reader might not be able to differentiate that 95% of the abstracts were generated - I think that would be reassuring. 68% is not that good and they even knew some of these abstracts were generated. So I think if someone came across the abstract in the wild, or if they were reviewing stuff, they might not realise that large language models have gotten so good at generating them and probably wouldn't think to think that it could be fake.

Chris - People are also raising concerns about, for instance, the use of tools like this to generate webpage content because, on the web, traffic is everything. Getting people to come to a resource, you throw adverts at them, you make revenue that way, you have a high foot fall site because you are creating content for your webpage. That was the bottleneck because that was where a person had to be involved and that's where money had to be involved.

Catherine - I think it gets to some very interesting questions about where do we go from here. In one way, could this be used in the hands of a responsible scientist to help take the burden off writing which sometimes can be, like you said, one of the bottlenecks of disseminating scientific work. Could it help improve equity across specifically scientists who have to write in a language that's not their own? What worries me also is, what if this technology is used for evil, right? There are these organisations that exist out there called paper mills that are basically generating scientific content for profit now with this technology that's so powerful, that's accessible and free. Could this be used by these nefarious organisations to spam science that's factually incorrect and dangerously convincing?

Chris - Well, could you go a step further and say, I've got a pharmaceutical company, it's not a very good one. It's deceitful and it wants to push a product. So what it does is generate hundreds of papers supporting a drug that it's invented, saying how good it is, encouraging real organisations to buy in, either investors or organisations who want to buy the drug or the product making money for that venture, when in fact it's all founded on fake science?

Catherine - The data that these models are trained on is detailed enough that it even knows the right range of patient cohort sizes to present in the generated results. For example, when we asked ChatGPT to write an abstract about study about diabetes, it included huge, huge numbers of patients beause a lot of patients have diabetes versus when we asked it to write an abstract about monkeypox, which is a much rarer, newer disease, it knew that the numbers needed to be much smaller. So certainly I think in the hands of these more nefarious or ill intended users, it could be a very dangerous technology.