Geoff Hinton: Why do we trust AI?

This episode of Titans of Science, Chris Smith spoke to AI pioneer Geoff Hinton about how AI works, and what we should be wary of...

Chris - The worry to me is that we regard what people say with a pinch of salt, some more than others. But we tend to have this enormous trust that we place in machines because they behave in a perfect way to our mind. And we are now using machines that behave more like people and have people's flaws in some respects as you've just been outlining to us. So are we going to have to educate people not to think about machines as quite so reliable in future?

Geoff - Yes. What we've produced in these big chatbots is like a new species that's very like us and very unlike a normal computer program, we have to learn not to treat the chatbots. Like you would've treated an old-fashioned computer program where you could rely on it. You can't.

Chris - When we were talking earlier, you said you started using computers to understand how the brain worked. But it strikes me that we are now at a position where computers and things like you've been describing are showing us how nature works. It's almost like the loop is closing.

Geoff - Yes. I mean I think we've understood a lot more about language from producing these big chatbots. So in the old days people like Chomsky said that language was innate, it wasn't learned. Well that's become a lot less plausible because these chatbots just start off with random weights and they learn to speak very good English just by looking at strings of English and learning. It told us a lot about how we work. We work very much like them, so we shouldn't be trusting them any more than you trust a person. We should probably trust them less than you trust a person.

Chris - When did you get this name of being the godfather of AI? Because we jumped straight into the hard stuff with our conversation. How did we get to the position we are in today? Because a lot of people suddenly think AI has arrived on the scene here and now, but you got your PhD in it not long after I was born. So what has happened in the last 40 something years and what's been going on in the background and what was your role in being so instrumental in it?

Geoff - Let me give you an analogy because there's another thing that happened in science that is actually quite similar. So in the 1910s or 1920s, someone who studied climate called Wegener decided that the continents had drifted around and that it wasn't just a coincidence that that bulge on South America fitted nicely into the armpit of Africa. They actually had been together and they came apart. And for about 50 years, geologists said, 'this is nonsense. Continent can't drift around. It's complete rubbish.' And Wegener didn't live long enough to see his theory vindicated. But in I think the 1960s or sometime like that, they discovered in the middle of the Atlantic there's this stuff bubbling up where the continents are moving apart and it's creating new stuff. And suddenly the geologist switched and said, 'oh, he was right all along.' Now with neural nets, something similar has happened. So back in the early days of neural nets, there were two kinds of theories of how you could get an intelligent system. One was you could have a big network of neurons with random connections in and it could learn the connection strengths from data. And nobody quite knew how to do that. And the other was, it was like logic. You had some internal language sort of like cleaned up English and you had rules for how you manipulated expressions in cleaned up English. And you could derive new conclusions from premises by applying these rules. So if I say Socrates is a man, and I say all men are mortal, I can infer that Socrates is mortal. That's logic. And most people doing AI, in fact almost everybody after a while, thought that that's a good model of intelligence. It turned out it wasn't. The neural nets were a much better model of intelligence, but it was just wildly implausible to most people. So if you'd asked somebody even 20 years ago, if you'd asked them, could you take a neural network with random initial connections and just show it lots and lots of data and have it learn to speak really good English, people would've said, 'no, you're completely crazy. That's never going to happen. It has to have innate knowledge and it has to have some kind of built in logic.' Well, they were just wrong.

Chris - When we hear that people put safeguards around AI, then is that where you've got a sort of barrier that it rubs up against as in you, you've got the freedom and the control of its connections in the way that you've been explaining. But when we want to say to it, 'no, I don't want you to invent black Nazis,' which is the problem we had before, it was coming up with all kinds of generated images, there was an example shown that the images were completely historically inappropriate or implausible and now that's been fixed, or allegedly has been fixed. How do you then lean on your system so that it doesn't make silly mistakes like that?

Geoff - You first train up a system on a lot of data and unless you've cleaned the data very carefully, the data contains unfortunate things. People then try and train it to overcome those biases. Sometimes they get a bit over enthusiastic. And one way to do that is you hire a bunch of people who get your chatbot to do things and then the people tell you when the chatbot does something wrong or the chatbot maybe makes two different responses and people tell you which is the preferable response. And now you train the chatbot a bit more. So it makes the preferable responses and doesn't make the other responses. And that's called human reinforcement learning. Unfortunately, it's often easy to get around that extra training. If you release to the public the weights of the neural network, then people can train it to overcome all that human reinforcement learning and start behaving in a racist way again.

Chris - But why is the system not bright enough, I'm using that word carefully and in inverted commas, to know that it's getting it wrong? Why does it not then go, 'well, hang on a minute there. There wasn't a particular group represented that I'm showing here, so that must be wrong. I'll correct that.' Why does it not self-correct?

Geoff - So probably initially before Google put a lot of work into getting it to be less biased, it wouldn't have produced black Nazis. But Google put lots of work into making it what it thought was less prejudiced, and as a result it started producing black Nazis. That's unfortunate, but you have to remember when it's producing a picture of a Nazi, it's not actually remembering a particular Nazi. It's just saying what it finds plausible.