Dr Colin Davis, Royal Holloway University of London
Chris - In last year’s general election we had the exciting spectacle of a leaders debate, and to help us get an idea of how well the speakers were doing, we could watch “the worm” - a real-time computer generated graph that showed how much a sub-set of the audience approved or disapproved of the comments the leaders were making. But could the worm itself, not just the speakers words, alter how we feel about the outcome? Professor Colin Davis from the Department of Psychology at Royal Holloway, University of London, has been trying to find out.
Colin - So the “worm” is something that has been around for quite a while. Initially it was used by politicians so that they could see for themselves how well their message was being received. Then, at some point, people in television thought, “well, this is interesting for the viewers as well, so we should give them access to this information.” The question that we were interested in was, what effect does the worm have on how viewers at home perceived the debate and what the candidates were saying?
Chris - So how did you actually do the study?
Colin - Well, we ran what was conceptually a very simple experiment, although technically it was somewhat difficult. We had two quite large groups of subjects come in on the evening of the final election debate last year (on April 29th), and they watched a version of the debate that includes the worm (the squiggly line going up and down). But we played a little trick on our subjects because although they were watching the genuine live debate, which we were getting from the BBC stream, the “worm” that they were seeing wasn’t the real worm, it was controlled by us. I was sitting in my office, watching the debate, and pressing some keys to move the worm about and hopefully making it look plausible. The worm that our subjects saw was based on the one that I was moving about, but biased in a particular direction. So for one group the worm was systematically biased in favour of Gordon Brown, and for the other group it was biased in favour of Nick Clegg. Then we used some video mixers so that we could superimpose our worms over the live BBC broadcast. Based on people’s responses afterwards via questionnaire, we can tell that our deception was successful, so the subjects on the whole believed that this was a real broadcast and the worm was genuine.
Chris - But more critically, what was the outcome when you ask the students who won the debate?
Colin - What our results suggest is that the worm is having a huge influence. In fact, it’s much greater than we had anticipated. Our two groups had completely different ideas about who had won the debate and their opinions were consistent with what the worm had been telling them. So the group that saw a worm which favoured Gordon Brown thought that he had won the debate, whereas the group that saw the worm which favoured Nick Clegg overwhelmingly thought that he was the winner. And more worryingly perhaps, we saw a similar, slightly smaller effect when we asked people about their choice of preferred Prime Minister. So if people had been voting immediately after this debate, it seems like our manipulation could have had a significant effect on how they voted.
Chris - If the worm is based on a sample size that is representative of the population as a whole then it shouldn’t be a problem. But if we look at what the BBC actually did when they generated the real worm, how many people did they ask to produce the data that we saw - how big was their sample size and how do we know it is a fair representation of public opinion?
Colin - Well that's particularly concerning. The BBC’s worm was based on only 12 undecided voters for each candidate and that's pretty typical. ITV had their own worm which was based on 20 undecided voters. CNN in the US used the worm for the presidential debates in 2008, and they had 30 voters for their worm. So these are very small samples, much smaller than those that you would ordinarily use for political polling. It could very easily be affected by a small number of outliers, and it could also be very easily deliberately manipulated.
Chris - I think the point that you've made in your paper is that the whole purpose of these TV debates is to try to present the leaders in an impartial way without any media spin and filtering. But you completely skew the debate by basing the worm on such a tiny sample size of just 12 people. Especially given that these people are probably recruited locally from nearby the TV station, you've got a terrible subselection problem, haven’t you?
Colin - Yes, you’ve really got no guarantee that you will have a representative sample and I think there's also some cause to be concerned about partisan forces attempting to manipulate it. For instance, The Guardian had their own worm (which they called a ‘sentiment tracker’) and while no one would claim that it was a representative sample, it’s interesting to note that the Liberal Democrats were caught trying to manipulate The Guardian worm. Staff from Liberal Democrat offices were using lots and lots of computers to vote in that. Beyond that though, the debate is supposed to be about giving viewers a chance to form their own opinions in a way that isn’t affected by media spin, and the worm is stopping them from doing that. When we see spin doctors come on after the debate and say, “Gordon Brown clearly won that debate”, we have the opportunity to think, “Well, clearly he’s biased.” But the worm is much more insidious than that and I think that's a real concern.
Chris - And just to finish off very briefly Colin, what would be your proposed solution? Should the media report where their worm samples come from and what size they are, so that viewers have some chance of making a judgment as to whether it can be relied upon? Or should we just ban it altogether?
Colin - Well, my preference would be to say that we shouldn’t have a worm at all. If there is going to be a worm, then I think it should be tightly controlled, and in the same way that there are quite strict rules about the opinion polls that you see in newspapers, there should be rules governing the worm. It should be very clear how it’s being created, what the sample is that they used, and so forth.
Chris - Fantastic, thank you. That's Professor Colin Davis from Royal Holloway, University of London. He’s published that work this week in the journal PLoS One.