Pitbull reviewers: threat or menace?
« previous post | next post »
Worth reading: Virginia Walbot, "Are we training pit bulls to review our manuscripts?", Journal of Biology 8:24, 2009.
I strongly support what Prof. Walbot has to say. One way to think about it: she's talking about the difference between the standards you ought to apply in order to believe something, and the standards you ought to apply in order to publish it.
Unfortunately, pit-bull reviewers are roughly balanced by cream-puffs, so that the refereed literature is simultaneously polluted by junk and bereft of some worthwhile stuff. Some glimmers of what might be a better system can be seen here and there on the web — arXiv.org is the best known — but given the difficulty of the problem and the cultural conservatism of the academy, change is likely be slow.
[Update: Peter McBurney has reminded me of a classic paper by J. M. Chambers and Agnes M. Herzberg ("A Note on the Game of Refereeing", Applied Statistics 17(3): 260-263, 1968). Here the authors introduce their model:
]
marie-lucie said,
March 16, 2009 @ 8:27 am
This problem was addressed a year or two ago in an article in the Canadian academic periodical University Affairs. One anecdote cited a review editor who had to drop a reviewer from his list for starting his reviews with "This is the worst paper I have ever read".
Mary Kuhner said,
March 16, 2009 @ 2:35 pm
I once received, in place of the expected two reviews on a paper, one (reasonably favorable) review and a note from the journal editor which said, roughly, "The second reviewer was apparently hostile to the concept of statistical inference; as the review was not useful, I am sparing you the annoyance of seeing it."
But most of my scientific review experiences have been okay, and some have been excellent; and in retrospect I even appreciate the reviewer who said, "I know there' s a concept of 'least publishable unit' but this paper falls short even of that" because, alas, he or she was entirely correct.
peter said,
March 17, 2009 @ 2:41 am
The economist and mathematician Graciela Chichilnisky at Columbia has written of her adverse experiences with editors and referees in those two fields, here:
http://www.chichilnisky.com/pdfs/papers/187.pdf
Eric Baković said,
March 17, 2009 @ 2:43 pm
Nature has a blog dedicated to the peer review process, which I found out about via this editorial on mentoring in the 14 June 2007 issue of the journal.
peter mcburney said,
March 22, 2009 @ 8:06 am
Another game-theoretic treatment of academic publishing is in this (serious) paper:
J. R. Faria [2005]: "The game academics play: editors versus authors", Bulletin of Economic Research, 57 (1): 1-12.