Naked Science Forum

Life Sciences => Physiology & Medicine => Topic started by: chiralSPO on 24/02/2020 20:56:33

Title: How bad can stats reporting be in mainstream media?
Post by: chiralSPO on 24/02/2020 20:56:33
CNN (I know... I bring it upon myself... it is one of many news outlets I get my info from) had an interesting article online today, concerning the usage rates of seniors (65+) using cannabis. (https://www.cnn.com/2020/02/24/health/weed-marijuana-seniors-wellness/index.html )

But the discussion of the statistics was SOOO BAD, I have to vent here!
In one excerpt:
Quote
In 2006, only 0.4% of seniors over 65 reported using marijuana products in the past year, they said. The newly published study found that by 2015, the number had doubled to 2.4%. By 2018, it had doubled again, with 4.2% of seniors over 65 using weed.

2.4% is not twice 0.4% (that's a sixfold increase), and 4.2% is not twice 2.4% (1.75x is not terribly far from 2x, butt still...)

But simple arithmetic issues are only some of it.

They go on to say (a few paragraphs later)
Quote
One of the most disturbing findings, he said, was an increase in cannabis use among older adults who also use alcohol. In 2015, only 2.9% of seniors reported both alcohol and cannabis use (although the data cannot say if they use simultaneously). By 2018 it had jumped to 6.3%.

Huh? If only 2.4% of seniors reported using marijuana in 2015, how did 2.9% use marijuana and alcohol? And again, if 4.2% of seniors reported using marijuana in 2018, how did 6.3% use marijuana and alcohol?

Presumably they are discussing different datasets or different studies or different definitions of "use" or something.... but come on! This is the most basic aspect of reporting science... no wonder people don't trust science!

(sorry for the rant)
Title: Re: How bad can stats reporting be in mainstream media?
Post by: chris on 24/02/2020 21:52:40
Agreed, those are pretty awful mistakes. Presumably they're just not checking numbers properly, or just cutting and pasting from various sources; usually it's people in a hurry to file something by a deadline and they don't go back and verify things sufficiently carefully.

The one I really like is when people say "the temperature has doubled, from 10 degrees C to 20 degrees C".

Nope.
Title: Re: How bad can stats reporting be in mainstream media?
Post by: chiralSPO on 24/02/2020 22:09:26
The one I really like is when people say "the temperature has doubled, from 10 degrees C to 20 degrees C".

Ahh... yes, I've seen that too. *facepalm*

I once had a student who tried to attribute % purity to a sample based on depressed melting point, by saying, "the experimentally determined melting point was 212 °C, and the reported pure compound had a mp of 218 °C, so it must be 97% pure"

I was like, where do I even start? You can't go from %deviation to %purity using melting point. And, there is the °C vs K issue... I was able to get the student to address both issues by having them calculate the freezing point depression of ice using colligative properties, and then having them calculate what percentage the mp should decrease by, and see if that lined up with the molality. they quickly saw that they could not divide by the mp of pure ice, and realized that they needed an absolute scale for division, and then saw that %purity and % mp change were not the same... LOL
Title: Re: How bad can stats reporting be in mainstream media?
Post by: chris on 24/02/2020 22:16:39
The thing is that unless someone takes the time to point these things out, like you did, we all make horrendous and heinous numerical mistakes before we known better. I know that my grasp of maths before I actually did A level maths was really very bad. Not that many people in the UK do A level maths, or at least they didn't when I was at school, so I suspect that includes a lot of scientists; unsurprisingly, they struggle with the numbers and the stats.
Title: Re: How bad can stats reporting be in mainstream media?
Post by: evan_au on 25/02/2020 09:22:18
I heard an influential radio announcer in Sydney talking about proposed legislation on the labelling of genetically modified foods.

The proposal was something like "You don't have to mention that the food is genetically modified if the percentage of genetically modified content is < 1%".

His criticism of this proposed legislation went like this: "What if the food has 10 ingredients? What about 20 or 50 ingredients?".
- His calculation clearly went: 20 ingredients, each 1% genetically modified produces a food which is 20% genetically modified... This justified his outrage!

PS: Doing a search now, it looks like the final legislation in Australia requires "genetically modified" labelling if the GM content is somewhere above 0.1% (or 1% if it is "accidental")...
http://www.aph.gov.au/DocumentStore.ashx?id=3761749f-12fc-4229-87e8-3dfa7d6fdd09
Title: Re: How bad can stats reporting be in mainstream media?
Post by: costarican on 01/03/2020 19:27:28
Sometimes they are in such a rush to get the news out, that numbers get fudged and then later corrected.