François Gonon et al., "Why Most Biomedical Findings Echoed by Newspapers Turn Out to be False: The Case of Attention Deficit Hyperactivity Disorder", PLoS ONE 9/12/2012:
Methods: We focused on attention deficit hyperactivity disorder (ADHD). Using Factiva and PubMed databases, we identified 47 scientific publications on ADHD published in the 1990s and soon echoed by 347 newspapers articles. We selected the ten most echoed publications and collected all their relevant subsequent studies until 2011. We checked whether findings reported in each “top 10” publication were consistent with previous and subsequent observations. We also compared the newspaper coverage of the “top 10” publications to that of their related scientific studies.
Results: Seven of the “top 10” publications were initial studies and the conclusions in six of them were either refuted or strongly attenuated subsequently. The seventh was not confirmed or refuted, but its main conclusion appears unlikely. Among the three “top 10” that were not initial studies, two were confirmed subsequently and the third was attenuated. The newspaper coverage of the “top 10” publications (223 articles) was much larger than that of the 67 related studies (57 articles). Moreover, only one of the latter newspaper articles reported that the corresponding “top 10” finding had been attenuated. The average impact factor of the scientific journals publishing studies echoed by newspapers (17.1 n = 56) was higher (p<0.0001) than that corresponding to related publications that were not echoed (6.4 n = 56).
This will not be a surprise to any honest working scientists, nor to members of the public who have observed the fate of science and technology in the media ecosystem over the years.
Gonon et al. focus on the role of publication bias and sensationalism at top scientific journals — but of course popular media have their own motivations, which often lead to credulous trumpeting of "results" that were never published in the technical or scientific literature at all, much less featured in a high-impact-factor journal. For a random sample of past LL posts on various aspects of these phenomena, see "Quit email, get smarter?" (4/23/2005), "The Agatha Christie Code" (12/26/2005), "It's always Silly Season in the BBC science section" (8/26/2006), "Flacks and hacks and brainscans" (11/23/2007), "David Brooks, Social Psychologist" (8/13/2008), "Debasing the coinage of rational inquiry" (4/22/2009), "The business of newspapers is news" (12/10/2009), "Texting and language skills" (8/2/2012), etc.
[Update — for some discussion of the within-science aspects of this issue, see Howard Wainer, "The survival of the fittists", American Scientist, September/October 2012.]