The new Freakonomics book is about to come out (called Super Freakonomics, natch), and Marginal Revolution thinks it's great: "a more than worthy sequel, a super sequel you might say." So does Bryan Caplan at econlog: "Overall, it's better than the original." Time Magazine thinks it's "very good — jauntier and more assured than their first".
But not everyone is convinced: negative voices include Ezra Klein, "The Shoddy Statistics of Super Freakonomics", WaPo, 10/16/2009; Matt Yglesias, "Journalistic Malpractice From Leavitt [sic] and Dubner", 10/16/2009; Bradford Plumer, "Does 'Superfreakonomics' Need A Do-Over?", 10/16/2009; Andrew Sullivan, "Not So Super Freak", 10/17/2009.
Ezra Klein sums up the general complaint: "The problem with Super Freakonomics is it prefers an interesting story to an accurate one." Specific points of contention include a shaky statistical argument in favor of drunk driving (they claim that it's safer than drunk walking), and an allegedly superficial and misinformed treatment of climate change.
I'll withhold judgment until I've read the book. But based on my experience with one particular story featured several times in Freakonomics columns over the past few years, Ezra Klein's evaluation rings true to me.
A few days ago, the NYT Freakonomics column featured (my Penn colleague) Justin Wolfers defending a study first promoted by Freakonomics in 2007 ("Nickled and Dimed by Barbara Ehrenreich", 10/14/2009). For my own take on the associated media circus, with an excessive number of additional links, see "The Happiness Gap is back is back is back is back", 9/20/2009. Summing it all up, at the risk of oversimplification, a 2007 academic study by Stevenson & Wolfers argued that
By most objective measures the lives of women in the United States have improved over the past 35 years, yet we show that measures of subjective well-being indicate that women's happiness has declined both absolutely and relative to male happiness. […] Our findings raise provocative questions about the contribution of the women's movement to women's welfare and about the legitimacy of using subjective well-being to assess broad social changes.
It's possible to quibble about how meaningful the female-happiness changes are — they're small relative to the yearly noise in the General Social Survey, for example — but I gather that Stevenson and Wolfers' questions are plausible ones for those who think that economic "utility" ought to translate straightforwardly into "happiness". However, from the beginning, the mass media presented these "happiness gap" results in a spectacularly misleading way. And by "mass media" I don't mean Glenn Beck and Rush Limbaugh. Rather, the guilty parties have included David Leonhardt, Steven Levitt, Ross Douthat, Maureen Dowd, and Arianna Huffington.
Barbara Ehrenreich accuses Arianna Huffington of reviving the Happiness Gap circus in order to use her site as "a launching pad for a new book by the prolific management consultant Marcus Buckingham", Find Your Strongest Life: What the Happiest and Most Successful Women Do Differently, by "[giving] Buckingham a column in which to continue his marketing campaign".
This shouldn't surprise us, it seems to me. Overall, the promotion of interesting stories in preference to accurate ones is always in the immediate economic self-interest of the promoter. It's interesting stories, not accurate ones, that pump up ratings for Beck and Limbaugh. But it's also interesting stories that bring readers to The Huffington Post and to Maureen Dowd's column, and it's interesting stories that sell copies of Freakonomics and Super Freakonomics. In this respect, Levitt and Dubner are exactly like Beck and Limbaugh.
We might call this the Pundit's Dilemma — a game, like the Prisoner's Dilemma, in which the player's best move always seems to be to take the low road, and in which the aggregate welfare of the community always seems fated to fall. And this isn't just a game for pundits. Scientists face similar choices every day, in deciding whether to over-sell their results, or for that matter to manufacture results for optimal appeal.
In the end, scientists usually over-interpret only a little, and rarely cheat, because the penalties for being caught are extreme. As a result, in an iterated version of the game, it's generally better to play it fairly straight. Pundits (and regular journalists) also play an iterated version of this game — but empirical observation suggests that the penalties for many forms of bad behavior are too small and uncertain to have much effect. Certainly, the reputational effects of mere sensationalism and exaggeration seem to be negligible.
[For an interesting take on the history of the freako trend within academic economics itself, see Noam Schieber, "Freaks and Geeks; How Freakonomics is ruining the dismal science" TNR, 4/2/2007.]
[Update — Andrew Gelman, who also hasn't read the book, strikes me as right on the money in this pre-review:
The interesting question to me is why is it that "pissing off liberals" is delightfully transgressive and oh-so-fun, whereas "pissing off conservatives" is boring and earnest? Based on their writings in Freakonomics 1 and their blog, Levitt and Dubner strike me as open-minded political pragmatists, so it's not that I think they have a big political agenda.
It's possible to write things that piss off conservatives while still retaining an edgy, transgressive feeling–take a look at Nate Silver (or, to take a less analytical example, Michael Moore)–but I think it's a little harder to do. Flouting liberal conventional wisdom is funner somehow. As I said, I think there's something more general going on here but I don't feel I have a full picture of this phenomenon.
Freakonomics 1 was based on Levitt's previous research, which was all over the map, whereas in Freakonomics 2 the authors got to choose ahead of time what to be counterintuitive about. […]
If they're not careful, this book will send them from the "popular science" to the "political punditry" category, with no turning back. Perhaps Freakonomics 3 will have a chapter explaining why evolution is just a theory, not actually proven at all?
I guess that it tells us something about the Zeitgeist that there's apparently no space for a left-wing Freakonomist. And maybe it tells us something about me, that I was taken slightly aback by Andrew's suggestion that Freakonomics had not been "political punditry" all along… (An update from Andrew is here.)]
[Update #2 — Paul Krugman has three posts on the global-warming chapter: "A counterintuitive train wreck"; "Superfreakonomics on climate, part 1"; and "Weitzman in context". His verdict, in three words: "snarky, contrarian games". Or at greater length:
… what it looks like is that Levitt and Dubner have fallen into the trap of counterintuitiveness. For a long time, there’s been an accepted way for commentators on politics and to some extent economics to distinguish themselves: by shocking the bourgeoisie, in ways that of course aren’t really dangerous. Ann Coulter is making sense! Bush is good for the environment! You get the idea.
Clever snark like this can get you a long way in career terms — but the trick is knowing when to stop.
And here's an in-depth discussion of the scientific and rhetorical issues by Joseph Romm, who is inspired to coin a new acronym:
In olden days, we called such folks Artistes of Bullshit, but now I’m gonna call them F.A.K.E.R.s — Famous “Authorities” whose Knowledge (of climate) is Extremely Rudimentary [Error-riddled? I'm still working on this acronym].
Another negative review of the climate stuff is here. I'm looking forward to some evaluations of the other chapters.]
[Update #3: Climate Progress on "It takes a village to debunk their anti-scientific nonsense, but why did they stop Amazon from allowing text searches?"; Brian Dupuis, "FAIL: Superfreakonomics", 10/17/2009; Brad Delong, "Six questions for Levitt and Dubner", 10/17/2009; P. O'Neill, "Freaky Gurls", 10/18/2009 (critique of the prostitution chapter); Gavin Schmidt, "Why Levitt and Dubner like geo-engineering and why they are wrong", RealClimate, 10/18/2009.
And Stephen Dubner's response to critics: "Global Warming in SuperFreakonomics: The Anatomy of a Smear", NYT, 19=0/18/2009. He gives a lot of explanations, but doesn't seem to me to address the core criticisms. Brad Delong's (earlier) exchange of emails with Dubner is here. Nate Silver weighs in here.]
[Email from Andrew Gelman: "Things get interesting when a scholar steps over the line and moves into pundit territory. All of a sudden the scholarly caution disappears. Search my blog for John Yoo or Greg Mankiw, for example…"
Brad Delong writes "*Sigh* Last Post on Superfreakonomics, I Promise", and concludes with " a little unsolicited advice for Levitt and Dubner. If I were them, I would abjectly apologize".
And Paul Krugman ("Superfreakingmeta") suggests an answer to Andrew Gelman's question about why pissing off liberals is fun while pissing off conservatives isn't:
Annoying conservatives is dangerous: they take names, hold grudges, and all too often find ways to take people who annoy them down. As a result, the Kewl Kids, as Digby calls them, tread very carefully when people on the right are concerned — and they snub anyone who breaks the unwritten rule and mocks those who must not be offended.
Annoying liberals, on the other hand, feels transgressive but has historically been safe. The rules may be changing (as Dubner and Levitt are in the process of finding out), but it’s been that way for a long time.
John Quiggin at Crooked Timber posted about "How SuperFreakonomics killed contrarianism". The discussion there reminded me of the role of Irwin, the secondary-school history teacher in Alan Bennett's play The History Boys, who teaches his students that in answering examination questions, “the wrong end of the stick is the right one”. In his memoir Untold Stories, Bennett describes his own discovery that the route to an Oxford scholarship was “the alternative journalism of a lowlier sort”, which attracts the interest of graders bored with mere competence by “turning a question on its head”.
Bennett describes his Finals at Oxford as “the last and most significant examination in my life, and it was in this examination that I cheated, just as I had cheated a few years before to get the scholarship that took me to Oxford in the first place.”
“I was not dishonest; I kept to the rules and didn’t crib, and nobody else would have called it cheating, then or now, but it has always seemed so to me. False pretences, anyway.”]