Worth reading
« previous post | next post »
Maria Brumm, "Giving the Lie: Blogs and Scientific Criticism", Green Gabbro, 4/6/2008. My favorite paragraph:
This cultural conflict been making the rounds of the geoblogosphere thanks to a pair of editorials in Nature Geoscience on the pros and cons of blogging. See RealClimate and Highly Allochthonous for summaries, Kim (who talks about fact-checking before teaching undergraduates), Chris again (with diagrams!), and James Annan for further discussion. (Incidentally: I am sure the various editors of the Nature Publishing Group are too dignified and professional to dance around their offices going "Oh yeah, baby, who controls the discourse? We control the discourse! UNGH!"… but when was the last time a blog post sparked a rambling article in the pages of EOS or Geology?)
Also worth the price of admission: the pointer to Steven Shapin's A Social History of Truth.
The current debate started with an editorial by Myles Allen in Nature Geoscience arguing that
criticism of peer-reviewed results belongs in the peer-reviewed literature. Direct communication over the Internet, far from creating a level playing field, just ploughs it up and makes the game impossible.
Prof. Allen came to this conclusion because he was slammed in print by a couple of journalists who accepted a year-old (and incorrect) criticism posted on a climate-oriented blog written by scientists. Apparently the post in question was neither corrected, nor linked to a rebuttal; and the journalists (what a surprise!) didn't choose to invest the few seconds of web search that it would have taken to find the rest of the story.
I've posted on both sides of this discussion: "Raising standards — by lowering them", 3/7/2005; and "Blogging from the seat of power", 7/22/2006.
Ironically, the Horrible Example that I complained about in "Blogging from the seat of power" was a blog-like column "To be blunt", written by Helen Pearson under the pseudonym "Sybil" — and published at news.nature.com.
So I sympathize with Prof. Allen's experience, but I'll disagree with his conclusion, for a set of simple and practical reasons.
The articles related to language that are published in journals such as Nature and Science are sometimes — in fact, often — inexpertly reviewed and deeply flawed. But linguists, including me, who have submitted criticisms of such articles are usually met with rejection. This is understandable. Paper and physical mail are expensive, and space is limited, and the same editors who accepted the flawed articles to start with are unlikely to be impressed by submissions questioning their judgment.
And in the specialist literature related to language, there are other issues (for example, reviewing cycles that are often longer than two years, as well as profound subdisciplinary and subcultural differences) that make genuine intellectual conversation difficult if not impossible.
So my opinion of the peer-reviewed literature is that it's the worst possible system for scientific communication, except for all the others. But let's not foreclose the opportunity to experiment with ways of making things better, just because an incorrect criticism by scientists in the blogosphere has been picked up by some careless journalists. After all, journalists pick up and publicize nonsense from the peer-reviewed literature all the time, or take good stuff and turn it into nonsense.
Meanwhile, there's one Language Log policy that I strongly recommend to all scientifically-inclined bloggers: whenever someone complains about a critical post, offer them the opportunity to respond. If they accept, post their response prominently, and link to their response in the original criticism. Example: "Groseclose and Milyo Respond" (8/2/2004), responding to "'Liberal bias', noch einmal" (7/5/2004). The background is described here.
(Of course, you can go on to respond to their response. In the cited case, I did so here and here; the authors in question declined to continue the conversation.)
[Note that the bloggers in question believe that Prof. Allen's description of the controversy is somewhat misleading — see "Blogs and peer-review", Real Climate, 4/3/2008:
Myles' piece, while ending up on a worthwhile point of discussion, illustrates it (in my opinion) with a rather misplaced example that involves RC – a post and follow-up on the Stainforth et al (2005) paper and the media coverage it got. The original post dealt in part with how the new climateprediction.net model runs affected our existing expectation for what climate sensitivity is and whether they justified a revision of any projections into the future. The second post came in the aftermath of a rather poor piece of journalism on BBC Radio 4 that implied (completely unjustifiably) that the CPDN team were deliberately misleading the public about the importance of their work. We discussed then (as we have in many other cases) whether some of the responsibility for overheated or inaccurate press actually belongs to the press release itself and whether we (as a community) could do better at providing more context in such cases. The reason why this isn't really germane to Myles' point is that we didn't criticise the paper itself at all. We thought then (and think now) that the CPDN effort is extremely worthwhile and that lessons from it will be informing model simulations some time into the future. Our criticisms (such as they were) were mainly associated instead with the perception of the paper in parts of the media and wider community – something that is not at all appropriate for a peer-reviewed comment.
]