Naked Science Forum

Life Sciences => Physiology & Medicine => COVID-19 => Topic started by: EvaH on 07/12/2020 14:43:51

Title: How is vaccine effectiveness calculated?
Post by: EvaH on 07/12/2020 14:43:51
Roger asks:

As I understand it, if the vaccine test on a large group results in 90% of the entire number of those contracting the virus being in the half that have not received the vaccine, this is called 90% effective. But if it had been 50:50, that would not be 50% effective, it would be 0% effective. So 90% indicates that the vaccine immunised 40 people out of 50 people, which is 80%. What am I misunderstanding please?

Can you help?
Title: Re: How is vaccine effectiveness calculated?
Post by: Zer0 on 27/03/2021 09:39:38
Hello Roger!
🙋

I did Not Understand your statement...

" But if it had been 50:50, that would not be 50% effective, it would be 0% effective. "

& Assuming you are Not registered on the TNS Forum, maybe i would never end up Understanding the statement.
👎

Perhaps this following link will clear out confusions based upon how Exactly is the
 " Vaccine efficacy or vaccine effectiveness " Calculated.
👍

https://www.cdc.gov/csels/dsepd/ss1978/lesson3/section6.html

Thanks & Credits - CDC.gov.

P.S. - Sorry 4 d Delay in Responding.
😷
🙏
Tc!
Title: Re: How is vaccine effectiveness calculated?
Post by: evan_au on 27/03/2021 10:24:14
Quote from: OP
his is called 90% effective
Clinical trials are designed to measure "efficacy": The chance that it does what it claims, under ideal conditions.

Once rollout starts, conditions are not so ideal. The chance that it does what it claims, under normal conditions is called effectiveness.

Similar concepts, similar words, but measured under somewhat different conditions.

Quote from: OP
But if it had been 50:50, that would not be 50% effective, it would be 0% effective.
Yes, that is correct.

It is also possible for the treatment arm in a Clinical Trial to be affected more severely than the patients in the placebo arm.
- When such conditions occur, the oversight committee recommends premature termination of the trial, to prevent more illness in the treatment arm.
- But such actions are only taken after enough adverse events are recorded - enough to give (say) 95% confidence that the new treatment is worse than no treatment (or the previous best treatment).

Quote
if the vaccine test on a large group results in 90% of the entire number of those contracting the virus being in the half that have not received the vaccine, this is called 90% effective
The number of adverse outcomes in each arm of the trial can almost be quoted from the raw figures in the trial
- If 90% of adverse outcomes were in the placebo arm, that is a good endorsement of a medication.

However, there is a bit of statistical manipulation of the figures before they quote an efficacy figure.
- Perhaps after some medical analysis and discussion about whether these adverse events were caused by the medication or external factors.
- Eg if there were 2 deaths in the treatment arm, and one was caused by getting hit by a truck, the manufacturer might argue that this was not caused by the medicine... Eliminating this patient from the count would make a big difference to their efficacy score!

A large clinical trial might cost $100 million.
- This is why pharmaceutical companies and regulators employ professional statisticians when designing and interpreting the results of clinical trials
- So they can obtain meaningful results from the trial (the "power" of the trial)
- And can estimate the probability that the result was obtained by mere chance
- Often, the conclusions would be stated as Efficacy=92% (with a range of 85% to 97% at p=.05)
- Where the p=0.05 says there is a 5% chance that the result might be outside this stated range.
- If you want a more accurate result (smaller range of uncertainty) you need more patients, it will cost more money and take longer to gain approval

See: https://en.wikipedia.org/wiki/Confidence_interval

But beware:
Quote from: Mark Twain
There are three kinds of lies: lies, damned lies, and statistics.
In the past, pharmaceutical companies have been accused of withholding publication of unfavorable results.
- That is harder to do in a pandemic, where the WHO maintains a list of all clinical trials, and everyone is looking for the results of every trial!
- Ideally, clinical trial results would be published in a peer-reviewed journal with all of the statistical details present and visible to readers. Some of the vaccine manufacturers have been accused of publishing results via press releases which are utterly devoid of the statistical facts behind them. Some of this behavior may be driven by stock-exchange rules?
- Astra-Zeneca has been accused of "marketing spin" in combining the results of several vaccine trials in ways that are not statistically rigorous.
Title: Re: How is vaccine effectiveness calculated?
Post by: Zer0 on 27/03/2021 17:34:05
@evan_au
🙋

Because of You, now i finally Understand what Roger's statement meant.
&
Again because of You, now i Know that his statement is logical & true.
👍👍👍

P.S. - There are 3 kinda Lies...
Lies, Damned Lies & Statistics!
Evan.
😇
🙏
(Just Wondering, Why do so many Questioners never join US directly on the TNS Forum?
Surely they have Alot more questions to ask besides just a single one, Right?)