Clinical U-turns

When you go to see the doctor, and they advise a course of treatment, do you ask them for the evidence behind their advice? Or do you just swallow it, alongside the pills they prescribe? Most of us inherently trust the medical profession and assume that the therapy we receive has been rigorously evaluated. Sorry to burst that bubble, but that’s just not the case. The vast majority of medical treatments are not supported by high quality evidence at all, just a good story and a spattering of biological plausiblilty. Which is why, speaking with Chris Smith, this man managed to find nearly 400 examples of doctors doing U-turns when some routine and mainstream therapies were actually subjected to proper scrutiny...

Vinay - Dr. VInay Prasad. I'm an associate professor of medicine here at the Oregon Health and Science University, and in this paper I'm on a mission to identify low value and no-value medical practices. Those are medical practices that offer no benefit to patients and only risks and costs.

Chris - So why would we dish out treatments like that for people?

Vinay - I think doctors have long been seduced by medical practices that seem like they should help patients for which we don't have confirmatory studies documenting that they actually do help patients. And some of the reasons we're seduced by these practices is they make a whole heck of a lot of sense biologically; they have a strong plausibility for why they should help; and also we are motivated by our sense of optimism: doctors want to be able to offer treatments to patients that benefit them. Finally, I think one of the other incentives that drives this problem is the financial conflicts in biomedicine. Many people make a great deal of money from recommending practices that may or may not help patients and they might find it difficult to evaluate those practices rigorously or to abandon them when they fail.

Chris - So how did you go about finding these reversals where where people have done a U-turn on what appropriate therapy should be. And can you give us some examples of the sorts of things that you flushed out?

Vinay - Yeah, absolutely. So we tackled this by a literature based review; we picked three high impact medical journals and we surveyed 15 years of biomedical publications and we looked for randomised trials that tested and contradicted established medical practices or things we were doing. But we didn't just stop there. We did a systematic review for every one of the topics that seemed like they might be reversals to make sure that the totality of the evidence really found that they didn't help patients. Some classic examples include the use of steroid injection for low back pain and spinal stenosis: a number of randomised studies find that when you compare a steroid injection to a saltwater injection both groups get better. But the steroid injection group does not get additionally better: it's a placebo effect.

We found that using a popular catheter called a swan ganz catheter it can provide you information about the hormone dynamics in the heart. But that information doesn't leverage improved he alth outcomes for patients with shock.

Another example that many listeners may be aware of is the use of stenting for chronic stable angina; stenting for stable angina is a multi-billion dollar a year industry and it's often done with the hope or the expectation on the patient's side that will lower the risk of a heart attack and improve mortality. And we found randomised control trials contradict both of those claims.

Chris - Now is this physician-led, or actually is it pushy patients, or is it both?

Vinay - There's certainly a case in biomedicine that patients sometimes ask for medical practices that doctors may have ambivalence towards. But I think in the practices that we look at in this data set, these were predominately developed, curated, recommended and extolled by physician bodies and physician groups. So it really is on the physician side. Doctors were optimistic that these were things that were gonna help our patients. Unfortunately in retrospect we turned out to be wrong.

Chris - When you were looking at these studies was there any obvious way in which this dogma got established incorrectly in the first place? Was there a sort of common pathway through which something inappropriate or ineffectual treatment actually gets established? Because that would be the point of intervention wouldn't it -the way in which we can prevent this happening if we know how it tends to happen?

Vinay - Yes. I mean I think the commonality here is that all of these practices got established based on weak, low, or quasi-experimental evidence. By that I mean by traditional hierarchies of evidence, we typically put randomised controlled trials at the pinnacle of evidence based pyramids and at the bottom we put case reports, uncontrolled observational studies, historically-controlled studies, or studies with poor control arms. And in almost all of these cases, what drove a practice to prominence were those sorts of lesser evidence. Studies that are not adequately designed to test whether or not an intervention is better than the best available therapy of the time. I think this has been one of the major challenges in the movement of evidence based medicine which is that now 30 plus years into this movement many of us had felt that the standards of adoption of medical practice would have improved over time. But some of us have been disappointed that they haven't improved that fast and we still have this enthusiasm for low levels of evidence.

Chris -  Very expensive though, isn't it, the kind of that you're seeking and that you feel comfortable with to accept a treatment works: you're asking for spends in the millions in order to establish that evidence base very often aren't you. And that's just not feasible. When a treatment is just getting going so there's got to be a start somewhere?

Vinay - I think that's a great question. One of the things is is that if you look at the average cost to enroll a patient on a randomised trial in the United States you might get a lofty figure, like 20 to 30,000 U.S. dollars. At the same time, we have randomised registry trials that are done for as little as 50 dollars per participant. That's a randomised study called taste. I think it's interesting to me that randomisation - something that essentially is cheap and easy - has a tremendous price tag largely because of the imposed bureaucracy of randomised trials. We hear a lot about innovation. One of the major things an innovator could do in the space of randomised trials is make them cheap and easy to deploy; and that is something that's within our power - we've done it before with this taste study - and we can expand that model I think to other domains. And the other side of the question is some of these products for which we're not doing randomised trials. They're not cheap either. They often cost a hundred thousand dollars per year of therapy and they have cumulative health care spending in the billions of dollars. Sometimes it might make more sense to run a 20 million dollar randomised trial than spend 500 million dollars per annum on reimbursing a product that you don't know actually helps patients.

Chris - I mean that's a good point and I can't dispute that. But the fact is it's who spends that money because when the government spends it to establish evidence and then they find that the treatment doesn't work they'll say well that's money that we've wasted. But when patients and insurance companies spend money in an in an insurance led system like the US for example, actually the loss is someone else's. It's not government money, it's not public money. So actually that might bias the situation?

Vinay - I 100 percent agree with you that many people in the health care space are thinking about their short term profits and revenues and there is sort of a tragedy of the commons going on here. But we have to remember that, even in our system in the United States and of course in your system in United Kingdom, the majority of health care expenditures are indirectly or directly borne by public payers. The largest payment - the largest source of payments in this country - is the Center for Medicare and Medicaid Services. And to a large degree private insurance in United States have been subsidised through governmental agencies through the subsidies we have for private health care insurance. And so, to some degree, we all pay for medical practices that are practiced and borne out of insufficient evidence.