The Naked Scientists

The Naked Scientists Forum

Author Topic: Can we trust all studies produced by reputable universities?  (Read 7094 times)

Offline LeeE

  • Neilep Level Member
  • ******
  • Posts: 3382
    • View Profile
    • Spatial
Link to an article reviewing a report produced by the University of Sheffield that has been used as justification by the government to raise duties and increase regulation of booze.

http://www.theregister.co.uk/2008/12/11/alcohol_pricing_sheffield_study/


 

Offline DoctorBeaver

  • Naked Science Forum GOD!
  • *******
  • Posts: 12656
  • Thanked: 3 times
  • A stitch in time would have confused Einstein.
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #1 on: 11/12/2008 15:22:39 »
It doesn't surprise me in the slightest. I've been privy to reports commissioned by the DoH regarding cannabis use. Many of those are just as dubious as this UoS report yet are cited as irrefutable evidence by the government.

There are certain ways of wording things that can be twisted very slightly to say what the report commissioners want it to say, yet still be vague enough that if the 5hit hits the fan the authors can say that wasn't they meant.

Isn't government funding a wonderful thing!

(Incidentally, it's not just government-commissioned reports. Pharmaceutical companies, among others, are just as guilty of demanding dodgy reports)
 

Offline LeeE

  • Neilep Level Member
  • ******
  • Posts: 3382
    • View Profile
    • Spatial
Can we trust all studies produced by reputable universities?
« Reply #2 on: 11/12/2008 23:59:42 »
I've seen some stuff like this before, but not quite to the extent that some of the conclusions directly contradict the data upon which the report was based (the degree of relevance of the data notwithstanding, for if the data is irrelevant then no conclusion can be drawn).

It used to bother me that I had a largely misanthropic point of view until I realised that any other point view required ignorance or denial.
 

Offline Don_1

  • Neilep Level Member
  • ******
  • Posts: 6890
  • Thanked: 7 times
  • A stupid comment for every occasion.
    • View Profile
    • Knight Light Haulage
Can we trust all studies produced by reputable universities?
« Reply #3 on: 12/12/2008 08:39:51 »
It's proof that you can twist and turn anything to say what you want it to say.

The glass is half empty.

No, the glass is half full.
 

Offline DoctorBeaver

  • Naked Science Forum GOD!
  • *******
  • Posts: 12656
  • Thanked: 3 times
  • A stitch in time would have confused Einstein.
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #4 on: 12/12/2008 12:49:42 »
Quote
I've seen some stuff like this before, but not quite to the extent that some of the conclusions directly contradict the data upon which the report was based

Yes, that does seem to be a bit obvious in this instance.
 

lyner

  • Guest
Can we trust all studies produced by reputable universities?
« Reply #5 on: 12/12/2008 21:02:37 »
It wouldn't be so sad if the government didn't fund all this research. They may as well just do what they wanted in the first place and spend the money on tanks or whatever.
 

Offline LeeE

  • Neilep Level Member
  • ******
  • Posts: 3382
    • View Profile
    • Spatial
Can we trust all studies produced by reputable universities?
« Reply #6 on: 13/12/2008 14:11:33 »
They shouldn't be doing what they want in the first place - they're supposed to be representing us, not ruling us.
 

Offline DoctorBeaver

  • Naked Science Forum GOD!
  • *******
  • Posts: 12656
  • Thanked: 3 times
  • A stitch in time would have confused Einstein.
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #7 on: 13/12/2008 14:51:46 »
They shouldn't be doing what they want in the first place - they're supposed to be representing us, not ruling us.

Dream on!
 

Offline Bored chemist

  • Neilep Level Member
  • ******
  • Posts: 8669
  • Thanked: 42 times
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #8 on: 13/12/2008 16:04:14 »
The purpose of the study was to obtain an estimate of the effects on health and crime of raising the price of alcohol.

It is not practical to do this by direct experimentation.

The best available data is not new or based on UK measurements.
However, given that people are much the same wherever you are and they don't change much from year to year the data is still the best we will get.

Governments (like everyone else)often have to act on limited data.
Since the aproprate experiment is impossible you can hardly blame the researchers for using the best available data. Meta-anlaysis is a recognised and respectable technique.

Why is this protrayed as bad science?

incidentally the last copy of the BMJ I saw carried a brief note explaining that the experiment was done (the other way round) in Finland (I think- I'm sure it was somewhere in Scandanavia)
The government reduced duty on alcohol.
There was a rise in alcohol related health problems, particularly among women and those on low incomes.

The UoS report's conclusion has been shown to be true for a different counrtry- why would you not expect it to apply here?

Is there any reason not to trust the university?
 

Offline DoctorBeaver

  • Naked Science Forum GOD!
  • *******
  • Posts: 12656
  • Thanked: 3 times
  • A stitch in time would have confused Einstein.
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #9 on: 13/12/2008 16:17:09 »
BC - the main problem that I can see (although I have to admit I have not seen the original report) is that their conclusions do not seem to be supported by the evidence they looked at.
 

Offline Bored chemist

  • Neilep Level Member
  • ******
  • Posts: 8669
  • Thanked: 42 times
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #10 on: 13/12/2008 17:24:55 »
There is nothing that I can see in the page cited above that says the report isn't correct, just that the evidence is old and mainly from elsewhere.
Without seeing the original report it is impossible to say if its conclusions are suported by  the evidence. However, if they were not then I doubt the reviewer would have spent time saying the data were old or inaapropriate. I'm sure he would have said, in big letters, that the evidnce didn't support the conclusions.
The best they seem able to do is say that one of the previous studies showed that the effect of price on consumption isn't that strong.
This may simply reflect a relatively small data set. They also say, about that set of data
"However, taxation changes are typically prompted by economic rather than public health considerations, and
Dee’s study cannot explain findings of many natural experiments of tax changes in the same
country, where it is highly unlikely that an “overnight change” of anti-alcohol sentiment has
taken place."
To me, that's a significant fault with the only study that says that raising taxes doesn't drop consumption.
Phrases like "the researchers grudgingly concedes that there may be what they call a "potential confounding variable" at work here" sugest to me that the reviewer doesn't know that its common for there to be confounding variables. Putting this perfectly valid technical term in quote marks seems to me to be an atempt to devalue it.
Anyway, unless I find time to read the whole 500 page report I guess I won't find out (I have had a quick look at it).
It hardly bothers me that a University has found in favour of what seems to be common sense as well as the experience of another country.
Perhaps the reason that they came to this conclusion was not because it was what the paymasters wanted- perhaps it's just true.
« Last Edit: 13/12/2008 17:58:35 by Bored chemist »
 

Offline DoctorBeaver

  • Naked Science Forum GOD!
  • *******
  • Posts: 12656
  • Thanked: 3 times
  • A stitch in time would have confused Einstein.
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #11 on: 13/12/2008 17:44:21 »
"Having taken this potential confounding variable into account, the effect of taxes on drinking disappeared,"

"There is low quality but demonstrable specific evidence to suggest that minimum pricing might be effective as a targeted public health policy in reducing consumption of cheap drinks."

Would you not say that there is a degree of contradiction in those statements? Or, if not actual contradiction, an amount of wordplay?
 

Offline Bored chemist

  • Neilep Level Member
  • ******
  • Posts: 8669
  • Thanked: 42 times
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #12 on: 13/12/2008 18:06:25 »
In one, and only one, study which tried to account for the effect of "anti alcohol sentiment" (however you might measure that) the effect disapeared. In the other studies the effect was demonstrated.

Here are the actual results that they cite in their report for price elasticity.

"Results
The price-elasticities for alcoholic beverages estimated in different studies have shown that
when other factors remain unchanged, an increase in price has generally led to a decrease in
alcohol consumption, and that a decrease in price has usually led to an increase in alcohol
consumption, with the size of the elasticities sometimes dependent on the relative presence
or absence of other alcohol policy measures (Farrell et al. 2003; Trolldal and Ponicki 2005).
Gallet (2007) reports median price elasticities for Beer (-0.36); Wine (-0.700); Spirits (-0.679)
and Alcohol (-0.497). An analysis of annual data from Australia, Canada, Finland, New
Zealand, Norway, Sweden, and the United Kingdom from the mid 1950’s to the mid 1980’s
found price elasticities of -0.35 for beer, -0.68 for wine, and -0.98 for spirits (Clements et al.
1997). This means that if the price of beer is raised by 10%, beer consumption would fall by
3.5%; if the price of wine was increased by 10%, wine consumption would fall by 6.8%; and if
the price of spirits increased by 10%, spirits consumption would fall by 9.8%. Whilst reported
price elasticities are consistently negative, there are differences between countries and within
countries over time, as to the degree to which alcohol consumers react to changes in the
price of alcoholic beverages. Even within a given country there may be a significant diversity
of price elasticity values cited across studies (Österberg 1995; Chaloupka, Grossman and
Saffer 2002). Reviews of demand models from 1989 and 1990 in the United Kingdom found
that the demand for beer, wine, and spirits was generally price-inelastic, with the demand for
wines and distilled spirits being more responsive to prices than the demand for beer (Godfrey
1989 1990). More recent estimates found price elasticities of -0.48 for beer consumed on
premises, -1.03 for beer purchased and consumed off premises, -0.75 for wine, and -1.31 for
spirits (Huang 2003).
The standard economic assumption in such studies is that tax changes were passed on to the
consumer in equivalent of price changes. This is a conservative estimate as empirical work
suggests that a 10% tax increase will usually generate a price increase of between 10 and
20% (Kenkel, 2005; Young and Bielinska-Kwapisz, 2002). Among the 91 studies included in
Wagenaar (2008), 74 found a significant negative relationship between taxation or prices and
consumption with an overall elasticity estimate of -0.51. Mean elasticities for specific
beverages were -0.46 for beer (105 studies), -0.69 for wine (93 studies) and -0.80 for spirits
(103 studies). The meta-analysis found significant relationships (p<.001) between alcohol tax
or price measures and indices of sales or consumption of alcohol (r = -0.17 for beer, -0.30 for
wine, -0.29 for spirits, and -0.44 for total alcohol). Significant effects were also found for
alcohol prices on heavy drinking, although the effect sizes were smaller than for overall
drinking (Wagenaar et al, 2008). Wagenaar’s estimates are quite similar to those from
another recently published analysis (Gallet, 2007) who reported median price elasticities for
wine (-0.70), spirits (-0.68) and all alcoholic beverages (-0.50), but a slightly higher elasticity
for beer (-0.36). The latter study included over 1000 estimates from studies conducted since
1945"
There are twenty odd values given for price elasticity.
Absolutely all of them are negative.
That's actually quite good evidence, rather better than a 1 in a million chance.

The person who wrote the web page cited above seems to have produced a much more biassed document than the University of Sheffield.

(BTW, I should point out that I live in Sheffield, but I'm not affiliated to the university in any way and I don't know any of the people who were involved in writing the repport.)
 

lyner

  • Guest
Can we trust all studies produced by reputable universities?
« Reply #13 on: 14/12/2008 21:41:56 »
They can't even get the results of strictly come dancing right so what hope have we of getting something like the alcohol thing right?
 

Offline DoctorBeaver

  • Naked Science Forum GOD!
  • *******
  • Posts: 12656
  • Thanked: 3 times
  • A stitch in time would have confused Einstein.
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #14 on: 14/12/2008 23:50:29 »
BC - in which case where does "Having taken this potential confounding variable into account, the effect of taxes on drinking disappeared," come from?
 

Offline testtest

  • Jr. Member
  • **
  • Posts: 11
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #15 on: 28/01/2009 13:07:35 »
Can we trust all studies produced by reputable universities?

In a word, NO! It isn't just about studies universities ARE doing for their government or corporate sponsors it's also about the studies they AREN'T doing in fear of upsetting any sponsors.

A brilliant article was written by CAFAS - The Council for Academic Freedom and Academic Standards titled "While Rome burns: Collusion with scientific fraud and the prognosis for disspassionate academic discourse".

The following is from newbielink:http://www.cafas.org.uk/update51.pdf [nonactive]

"The Battleground
Universities exist for only one reason: to add to human knowledge and to disseminate that knowledge through publication and teaching. While our unions are fighting important battles over academic pay we are starting to lose the battle over what it means to be an academic and the raison d'ętre of a university. The chief battleground of this war is in medicine.

The factors underlying the dubious role of medicine in this struggle are no mystery - pharmaceutical, biotech and agricultural companies have managed to develop an extraordinary stranglehold over academic life, scientific journals, governments, universities and common sense(1-6). It is therefore also no surprise that important recent books on the corporate corruption of academic discourse are devoted in large part to biomedicine."

I urge all of you to read the entire piece especially those of you who believe universities are places of free thought.

Regards.

« Last Edit: 28/01/2009 13:57:43 by testtest »
 

Offline DoctorBeaver

  • Naked Science Forum GOD!
  • *******
  • Posts: 12656
  • Thanked: 3 times
  • A stitch in time would have confused Einstein.
    • View Profile
Can we trust all studies produced by reputable universities?
« Reply #16 on: 29/01/2009 11:59:22 »
I shall read that later. Thank you for the link.
 

The Naked Scientists Forum

Can we trust all studies produced by reputable universities?
« Reply #16 on: 29/01/2009 11:59:22 »

 

SMF 2.0.10 | SMF © 2015, Simple Machines
SMFAds for Free Forums
 
Login
Login with username, password and session length