What IQ Tests Can't Tell You

Very few people today deny that intelligence is shaped by environmental factors as well as by genes, but the orthodox view has been that, at least in a developed country like the UK or America, what we mean by "intelligence" is basically the same over time. But Professor Jim Flynn has published a new book, "What is Intelligence?", that challenges this view.

Here is an example of a question found in the IQ test Raven's progressive Matrices. On the left it shows a design with a piece missing, and on the right six pictures, one of which is the missing piece. You have to select the piece that fits, which involves noticing similarities and differences across the rows and down the columns of the design. Raven's Progressive Matrices was once hailed as immune from cultural bias because it demands that you think out problems on the spot without a previously learned method for doing so. Flynn's work, however, has shown that cultural differences between times and places nevertheless radically affect its scores. Answer: The picture on the bottom right. First we notice that the missing piece must be a square. Then we see that the bottom half of the square must be solid black. The top half must be divided two sections (one fewer than the top half).

The story goes back to 1984 when Flynn, a social scientist from the University of Otago, New Zealand was sent some data by a Dutch psychologist P.A. Vroon, which Flynn now describes as the "bombshell in a letter box". The data showed that in a single generation, Dutch males had made enormous gains in an I.Q. test called "Raven's Progressive matrices", an example of which is shown on the right. When he saw this data, it struck Flynn that "if I.Q. gains occurred over time anywhere, they might have occurred everywhere," and after 25 years spent proving this was indeed the case he has now begun to work out why.

I.Q. scores have been rising steadily, by about 3 points per decade, ever since they were first administered. This is known as the Flynn Effect and it means that if we take the average teenager of today with an I.Q. of 100 and project the trend back to the 1900s, the average I.Q. would have been somewhere between 50 and 70. An I.Q. of 70 or below usually marks a mental disability. So, if I.Q. gains are in any sense real, "we are driven to the absurd conclusion that a majority of our ancestors were mentally retarded".

Within a widely used intelligence test (Weschsler Intelligence Scale for Children - WISC) one particular category, "similarities", has seen enormous gains (24 points between 1947 and 2002). Similarities tests ask questions such as "what do dogs and rabbits have in common?" Today it would seem obvious to answer (correctly for the purpose of an I.Q. test) that they are both mammals. However, a person from the early twentieth century would have been more likely to answer, "you use dogs to hunt rabbits". The correct answer assumes that the only pertinent reference in the world is the scientific classification.

But our ancestors were grounded in the concrete world around them and preferred a classification that reflected this: those based on functional relationships (this eats that) or similarities in appearance (this looks like that). Moreover we have developed new "habits of mind" such as taking hypothetical situations seriously, which further boost I.Q. scores. Flynn adds, "If you argued with a racist in 1900, and asked, 'how would you feel if your skin were black?' he would say 'don't be absurd, no one's skin changes colour.' Today, racists would feel that they had to take your questions seriously - and realize that you were challenging the logical consistency of their beliefs."

Differences in logic are not only found between time periods but also between cultures. It was a long held belief that illiterate or isolated peoples were unable to solve syllogistic problems. Scribner, however, did a study in 1977 with two tribes from Liberia and proved that this wasn't the case. She asked a Kpelle tribesman the following problem:

All Kpelle tribesmen are rice farmers.

Mr Smith is not a rice farmer.

Is he a Kpelle man?

This may seem like perfectly logical question to which it would be equally valid to answer "no". But to the tribesman it was completely unreasonable to ask this. He replied "If you do not know a person, and a question comes up, it's hard for you to answer. Because I don't know 'Mr Smith' I can't answer the question." This answer is equally valid but incorrect for the purpose of an IQ test. Again, the tribesman was grounded in the concrete world around him where conclusions are drawn based on everyday experiences. So the logical answer to give depends on your cultural background.

In order for an I.Q. test to give an accurate score it must be standardised using a representative sample of the age group, giving the median person a score of 100. If the subject is marked according to the wrong group, (e.g. one from the past or one biased to include too many people with little education) then the standards are obsolete. It becomes too easy to achieve the average score of 100 and the I.Q. is falsely inflated. Far from being a mere intellectual curiosity, not accounting for the Flynn effect gives rise to some serious implications: none more so than the particular case of criminals sentenced to the death penalty. In most states in America those convicted of murder are exempt from capital punishment if the defendant suffers from a mental disability, which ideally involves demonstrating poor adaptive functioning and an I.Q. score of less than 70. The consequences of obsolete norms inflating the I.Q. score from below to above this threshold place the prisoner at greater risk of execution.

So, I.Q. gains do not imply a lack of intelligence on the part of our ancestors, but that doesn't mean that they are trivial. They represent the spreading of the scientific ethos and mass education. As society has asked new things of our minds, they have adapted as a consequence. I.Q. tests are an indicator of the cognitive history of our time: rather than measuring the quality of a person's mind, they show how modern the society that they live in is.