Study: hacks often bamboozled by flacks

« previous post | next post »

Steven Woloshin et al., "Press Releases by Academic Medical Centers: Not So Academic?", Annals of Internal Medicine, 150(9): 613-618:

Background: The news media are often criticized for exaggerated coverage of weak science. Press releases, a source of information for many journalists, might be a source of those exaggerations.

Conclusion: Press releases from academic medical centers often promote research that has uncertain relevance to human health and do not provide key facts or acknowledge important limitations.

There's no indication that any of the 200 press releases in their sample was as bad as the infamous "Nobler Instincts Take Time" document from USC that set off a flurry of headlines like "Twitter makes users immoral, research claims". (For details, see "Debasing the coinage of rational inquiry: a case study", 4/22/2009.)  And even further out on the long tail of hooey are the virtuoso flacks who create scientific-seeming news out of little or no research at all.

But Woloshin et al. do conclude that investigators should "review releases before dissemination, taking care to temper their tone (particularly their own quotes, which we often found overly enthusiastic)". This is certainly good advice, though it's not much more likely to be followed than any other good advice that runs counter to its recipients' interests.

Ben Goldacre ("Dodgy academic PR", The Guardian, 5/30/2009) focuses on the role of the economic dynamics of the news business:

Obviously we distrust the media on science: they rewrite commercial press releases from dodgy organisations as if they were health news, they lionise mavericks with poor evidence, and worse. But journalists will often say: what about those scientists with their press releases? Surely we should do something about them, running about, confusing us with their wild ideas?

Now you may be inclined to think that a journalist should be capable of doing more than simply reading, and then rewriting, a press release: but we must accept that these are troubled times. Through our purchasing behaviour – and I assume someone cleverer than me measures these things competently – we have communicated to newspapers that we want them to be large and cheap, more than we want them to be adequately researched.

So in this imperfect world, it would be useful to know what’s in academic press releases, since these are the people of whom we are entitled to have the highest expectations. A paper in the Annals of Internal Medicine this month shows very clearly that we have been failed.

Was coverage of science significantly better in less journalistically troubled times, when newspapers were highly profitable? I'm not sure — I suspect that in mass media in general, sensation and entertainment have always been better business than sober and balanced presentation of research results.  And in the same way, the forces governing academic PR don't reward researchers for "tempering their tone".

The best thing, it seems to me, would be to enrich the journalistic ecosystem with more species in niches like the one that Goldacre's Bad Science column occupies — agile, razor-clawed predators culling the herds of science-news herbivores that graze the green shoots of press releases on the endless media plains.

Share:



5 Comments »

  1. Does this really happen? — Jill A. Smith said,

    May 30, 2009 @ 9:22 am

    [...] on Language Log, Mark Liberman points to a recent study regarding the somewhat dodgy intersection of science, [...]

  2. mgh said,

    May 30, 2009 @ 10:27 am

    Their study's coding as "exaggerated" any claim about animal models of disease having an "implication" for, or "in the future [...] could" lead to, future human therapies, seems — well — exaggerated.

    [(myl) This press release (!) cites a BMJ study whose conclusion is described this way:

    Animal studies are generally of poor quality and lack agreement with clinical trials, which limits their usefulness to human health, say the authors. This discordance may be due to bias, random error, or the failure of animal models to adequately represent clinical disease.

    The press release doesn't link to the cited paper, and doesn't even tell us who the authors were, so take it with a grain of salt :-). But really, I think the rate of translation from suggestive animal results to human therapies is so low that "exaggerated" is probably not too strong at all, as a description of any story implying that consequences for humans are likely. ]

  3. Sili said,

    May 30, 2009 @ 2:16 pm

    Well, if only we could get you out of the ember and into wild, we'd pretty much have released a nice little heard of velociraptors.

  4. mgh said,

    May 31, 2009 @ 1:23 am

    I think the rate of translation from suggestive animal results to human therapies is so low that "exaggerated" is probably not too strong at all, as a description of any story implying that consequences for humans are likely

    Mark, I'm ambivalent. Let's say you work on circadian rhythms in fruit flies (not my own field). You publish a paper identifying a gene that links the circadian cycle to sleep patterns in flies. Should you a) not write a press release, b) write one strictly about sleep behavior in flies, c) write one describing the result and saying something like "this finding may have implications for future treatment of sleep disorders in humans," or d) give up this basic science mumbo-jumbo and work on cancer.

    Most people go with c), but this study (and you?) would code it as an exaggerated claim. The flip side of the coin is that it is scientists' responsiblity to explain how basic research connects to human disease, so you don't get people like Sarah Palin saying fruit fly projects "really don't make a whole lot of sense" and "have little or nothing to do with the public good."

  5. Aaron Davies said,

    June 8, 2009 @ 12:10 am

    A paper [...] shows very clearly that we have been failed.

    I would like to nominate this sentence for the most-bizarre-use-of-the-passive-voice award.

RSS feed for comments on this post · TrackBack URI

Leave a Comment