Paying the price

A set of 149 papers that were retracted due to misconduct had been funded to the tune of $58m by the NIH.

Dr Chris Smith spoke to Ferric Fang from the University of Washington about not only the financial impact, but also the damage it can cause to the reputation of science overall...

Chris - All scientists follow a code of ethics and good practice, which says that the results that they publish must be true and accurate. Regrettably, we know that this isn't always the case. So, how much does scientific fraud actually cost? Ferric Fang has been looking at the impact in the US.

Ferric - With discussions of research misconduct, there's often an interest in how much public money has been wasted in research that turned out to be fraudulent. There's a case going on the in States right now involving a researcher who worked on an HIV vaccine project, and the laboratory involved had received almost 20 million dollars US in grant funds so I think the public is rightly concerned about whether their investment in science is being misspent and to what extent that's taking place.

Chris - How did you do that? How can you put a finger on work that's fraudulent? What are you using as your index of falsification?

Ferric - We looked at papers that had been retracted and we looked at the underlying causes for that. We then looked at what the grants were, that supported that work and we looked at how many other papers were supported by those grants, and then we apportioned the fraction of the grants that were attributable to that specific paper that turned out to be fraudulent and was retracted.

Chris - Just put some numbers on this for us. How many retractions were there? How big was your data set?

Ferric - The number of retractions ever in the research literature is actually pretty modest, a little over 2000 papers that had been retracted out of well over 20 million. Then we had to narrow those down further, so we were talking about articles that originated from the United States, and so, the subset of data that we were looking at ended up being 291 articles published between 1992 and then 20 years later, 2012.

Chris - As a percentage, that means that scientists are relatively well behaved, or at least put another way, when they're caught, the numbers seem to suggest they're relatively well-behaved.

Ferric - Well, I wouldn't dispute that statement. I do think that the vast majority of scientists are scrupulous and honest, but I also would say that the number of papers that are retracted for misconduct, which is only about 1 out 20,000, is really just the tip of the iceberg because there are many, many cases of misconduct which do not result in a retraction.

Chris - If you were to ask scientists to be candid and face no consequence but say to them, "Have you ever falsified data?" What proportion of scientists say then that they have?

Ferric - So, that exactly, has been done, and taken collectively, they suggest that on the order of 1 to 3 percent of scientists admit to having committed serious fabrication or falsification of data at least one time in their careers.

Chris - So that number's probably more realistic as a measure of the degree of misconduct. So, taking your present study, you're able to now put some kind of price tag, some kind of consequences on this. So, what did you find was the sort of financial impact? How much have we spent on research that turns out to be dud?

Ferric - The actual amount is really a very small amount. We came up with about 58 million dollars US out of billions of dollars that have been spent on research over that period of time. If you were to say we have to fudge that upward because 1 to 2 percent of scientists have actually committed misconduct, which I think would be a gross overestimate, we still end up with on the order of 1 and 1/2 percent.

Chris - Are you factoring in the snowball effects, because if there's some dodgy data in there and someone else then uses that to do legitimate work, then their legitimate work could be made illegitimate and therefore, a waste of money owing to the fact that it was founded on fallacious principles.

Ferric - So, you're absolutely right, but we did not try to factor that in. I think it would be very difficult, but this is one of the great concerns about fraudulent research. A good example is the research that was supposed to link vaccines with autism and intestinal disease. That paper has now been discredited, but the anti-vaccine movement continues to cite this kind of work as a foundation for its concern about dangers of vaccination. And so, there is a problem once a paper is in the literature and people believe it and they cite it and rely on it, and simply putting the word retracted over it, doesn't negate all of the impact that that work has had and the possible wasted effort that might be spent chasing blind alleys.

Chris - What about the flipside of this, which is that we assume by your reasoning, that all of the data in a paper is fraudulent? We take the case of Woo Suk Hwang, the Korean stem cell scientist, who famously had done all kinds of nefarious things with data falsification. Actually, some of his work was extremely good and absolutely sound. So, are we therefore, potentially, over-egging this pudding to a certain extent with the interpretations you're making.

Ferric - So, I think the concrete results of this study are that the actual dollar amounts that can be directly attributed to research that was fraudulent, are fairly modest, and they're not the major cost of research misconduct. I don't really think that this is the major cost. I think then we can move on to try to look at these costs of research misconduct that are much more difficult to put a number on but are much more serious. One of the things that we haven't really touched upon is what happens to the reputation of science? And I think this is of a great concern to scientists, because we only perform science because of public support for science and confidence that scientists are doing their best to create new knowledge that can be used for society's benefit. The damage done by people wasting time, going down the wrong direction, not knowing what's true anymore, I think, can be very substantial.

Chris - Ferric Fang from the University of Washington.