The Happiness Gap is back is back is back is back

« previous post | next post »

What may be the most widely-discussed statistical over-interpretation in history is coming around for the third time. The first gust front of commentary blew in with David Leonhardt in the NYT Business Section in September of 2007, echoed a few days later by Steven Leavitt in the Freakonomics blog. In May of 2009, Ross Douthat's NYT column recycled the same research for another round of thumb-sucking. And the same material has just been promoted again by Arianna Huffington ("The Sad, Shocking Truth About How Women Are Feeling", "What's Happening to Women's Happiness?", etc.), with an assist by Maureen Dowd ("Blue is the New Black", 9/19/2009).

Ms. Huffington tells us that

According to study after study, women are becoming more and more unhappy. This drop in happiness is found in women across the social and economic landscape. It doesn't matter what their marital status is, how much money they make, whether or not they have children, their ethnic background, or the country they live in. Women around the world are in a funk.

And it's not because of the multitude of crises we are facing. Women's happiness has been on a downward trend since the early 1970s, when the General Social Survey, a landmark study, began examining the social attitudes of women and men — who, by the way, have gotten progressively happier over the years.

MoDo chimes in:

According to the General Social Survey, which has tracked Americans’ mood since 1972, and five other major studies around the world, women are getting gloomier and men are getting happier.

Before the ’70s, there was a gender gap in America in which women felt greater well-being. Now there’s a gender gap in which men feel better about their lives.

People love this story. They love to speculate about the reasons for the trend — the favorites are variants of "too much feminism" and "not enough feminism" — and to tell us about their own happiness or lack thereof. Tens of thousands of readers, across the repeated reprises of this story in the mass media, have commented on various newspaper and weblog sites.  In a certain sense, this tidal wave of response validates the story, which clearly resonates with something in the spirit of the times.  But in fact, the empirical basis for all this fuss is so thin as to be practically non-existent.

I'll focus on the General Social Survey results, since I've looked into them in detail, but the rest of the worldwide background is similar.

One way to see what's happening is to look at this graph of General Social Survey results, from the preprint that kicked it all off in 2007:

Those who prefer tables may like to see it this way (taken from an earlier post on the subject):

If we sum up all the GSS responses across years, we get these proportions of answers to the question "Taken all together, how would you say things are these days — would you say that you are very happy, pretty happy, or not too happy?"

Very happy Pretty happy Not too happy
Male 31.2% 56.7% 12.1%
Female 32.4% 55.1% 12.5%

In the responses for 1972, 1973, and 1974 (the earliest dates available), the overall proportions were:

Very happy Pretty happy Not too happy
Male 31.9% 53.0% 15.1%
Female 37.0% 49.4% 13.6%

In the responses for 2004, 2006, and 2008 (the most recent dates available), the proportions were:

Very happy Pretty happy Not too happy
Male 29.8% 56.1% 14.0%
Female 31.2% 54.9% 13.9%

The best way to describe this, I think, would be to say something like:

In the early 70s, women self-reported their happiness at levels somewhat higher than men did. Specifically, 5.1% more of the women reported themselves "Very happy", while 1.5% fewer reported themselves "Not too happy".

30-odd years later, in the mid 00s, women's self-reported happiness was closer to men's, though it was still slightly higher. 1.4% more of the women reported themselves "Very happy", while 0.1% fewer reported themselves "Not too happy".

To Arianna Huffington, this means that "women are becoming more and more unhappy", while "men … have gotten progressively happier over the years". To Maureen Dowd, this means that "Before the ’70s, there was a gender gap in America in which women felt greater well-being. Now there’s a gender gap in which men feel better about their lives."  Ross Douthat described these numbers with the generalization "In postfeminist America, men are happier than women."

All of these statements are either false or seriously misleading.  Maybe, if you look at the data through a sophisticated statistical model, you can support a conclusion about the relative signs of the long-term-trends for males and females.  But any way you slice and dice it, there's not much there there.

I've cited the earlier stages in this discussion as motivation for a moratorium on using generic plurals to describe small statistical differences.  The contributions of Arianna Huffington and Maureen Dowd are, if anything, even better arguments for this (hopeless) cause.

Past LL happiness-gap posts:

"The 'Happiness Gap' and the rhetoric of statistics" (9/26/2007)
"Gender-role resentment and Rorschach-blot news reports" (9/27/2007)
"Why are economists so misleading?" (10/1/2007)
"The gender happiness gap: statistical, practical and rhetorical significance", 10/4/2007
"The happiness gap returns" (7/26/2008)
"The happiness gap is back" (5/26/2009)
"Women's happiness and pundits' accuracy" (5/27/2009)

Also maybe relevant, if you're not completely sick of the whole topic:

"Myth is truth (p < .05)" (12/23/2007)

If you want to do your own modeling, a csv file of the GSS happiness answers is here (some background on the data is here).

[I also note a certain lack of journalistic courtesy — Douthat didn't mention that Leonhardt and Leavitt had covered the same material a year and a half earlier, and Huffington doesn't mention Douthat, Leavitt or Leonhardt.  I know that journalists don't need footnotes, but if a pundit reprises a story that's previously been featured by other pundits, doesn't journalistic etiquette suggest a tip of the hat to the earlier authors? Dowd does cite Huffington, which seems like the normal practice.]

[Update 9/30/2009 — a column on this topic by Katha Pollitt, "Are you happy?", The Nation, 9/30/2009.]

[Update 10/14/2009 — and another by Barbara Ehrenreich, "Are women unhappier? Don't make me laugh", LA Times, 10/14/2009; with a response by Justin Wolfers, "Nickled and dimed by Barbara Ehrenreich", Freakonomics (NYT), 10/14/2009.]



26 Comments

  1. Benjamin Zimmer said,

    September 20, 2009 @ 1:31 am

    Dowd might have her own reasons to be extra cautious in her pundit-citing.

  2. D.O. said,

    September 20, 2009 @ 2:24 am

    In a certain sense, this tidal wave of response validates the story, which clearly resonates with something in the spirit of the times.

    Do you mean self-fulfilling prophecy or something else?

    [(myl) If you publish a story about how the streets have come to be paved with cheese, and rather than respond "no they're not", millions of people enthusiastically discuss whether cheddar or brie is more common, and whether this means that agricultural subsidies are too large or too small, then you've obviously gotten something right, even if it's not the facts.]

  3. Troy S. said,

    September 20, 2009 @ 2:38 am

    On the topic of misinterpreted science, I noticed the "Xerox machine" study you referenced in "Generalization and Truth" cited in a Sports Illsutrated recently. Say what you will about the Zeitgeist, I think people just like to cite factoids.

    [(myl) Yes, that factoid is a perennial favorite. Its long-term life is assured by the fact that it's now cited in hundreds of self-help books and textbooks, and no doubt in tens of thousands of business-education seminars.

    But not all factoids are equally successful, and the emerging science of factoidology aims to explain why. (Or would, if it existed.)]

  4. Ian Tindale said,

    September 20, 2009 @ 6:25 am

    Did they even bother to interview any dead Atlantic Salmon?

    [(myl) Not as far as I know. Nor, for that matter, does the GSS demographic sample include any other fish species. Unfortunately, the cost of using fMRI to determine the responses would have been prohibitive, never mind the fact that fMRI wasn't invented until 1992. But it's true, the GSS could have used cheaper methods like EEG or GSR. Definitely a missed opportunity, since you've got good headline opportunities any way it turns out ("Even gravlax get the blues"…).]

  5. D.O. said,

    September 20, 2009 @ 3:52 pm

    This irrelevant trend is not as obviously wrong as chseese covered streets for those who do not read GSS reports. And people are always interested in sex, drugs and politics. Folks just love to discuss gender differences and politics no matter what got them started. No Zeitgeist here. It might be that the commentariat needs factoids to start talking and this is the "spirit of our times".

    [(myl) Exactly. See "Bible Science stories" for more discussion of this point.]

  6. nemryn said,

    September 20, 2009 @ 4:31 pm

    What I'm getting from that data is, both men and women (thus, people in general) are less 'Very Happy' and more 'Pretty Happy'. I haven't done a formal analysis or anything, but that looks like a stronger effect than any gender difference.

    [(myl) By the method of interocular trauma (what strikes the eye), this is pretty clearly correct.]

  7. Bloix said,

    September 20, 2009 @ 11:47 pm

    Back in May I demonstrated to my own satisfaction, if no one else's, that, assuming these data are accurate, it's highly likely that men are actually more unhappy than women. Why? Because men are incarcerated at much higher rates than women – almost 2% of the male population is in jail or prison at any one time. If you assume (1) that the interviewers didn't have access to people in jail or prison and (2) incarcerated people are very likely to be unhappy (a fair assumption, don't you think?) then the data are simply incomplete. Adding the unhappy prisoners would push the rate for men above that for women. See comments here: http://languagelog.ldc.upenn.edu/nll/?p=1456

    Now, you may say, people in jail and prison don't count! And I say, why don't they?

    What you're really saying is, people in prison aren't the kind of people Maureen Dowd is talking about. And I say, that's true – and this data isn't gathered from the kind of people Maureen Dowd is talking about. It's not a survey of Times readers and their well-off college friends living in large metropolitan areas, and you can't draw conclusions about such people from this survey.

  8. Bloix said,

    September 20, 2009 @ 11:51 pm

    Clarification – almost 2% of the adult male population is incarcerated, not the total male population. As this survey is limited to adults, the figure is the correct one.

  9. Bloix said,

    September 20, 2009 @ 11:58 pm

    PPS- I understand that I'm off-point here – you're talking about drawing unwarranted conclusions from data, while I'm talking about crappy data. But I do think it's worth pointing out how entire populations are typically ignored in these discussions about what men and women do and think.

    [(myl) Let's take your point at full face value — that 2% of the adult male population has been left out of the GSS, due to being incarcerated; that the comparable proportion of the female population is small enough to be disregarded; and that the whole incarcerated population would report itself in the lowest happiness category, namely "not too happy". Then instead of the percentages given in the last table above, we'd have males over the past few years self-reporting at 29.2% "very happy", 55.0% "pretty happy", and 15.7% "not too happy", compared to females self-reporting at 31.2% "very happy", 54.9% "pretty happy", and 13.9% "not too happy".

    This would slightly increase the statistical falsehood of Ross Douthat's "In postfeminist America, men are happier than women" and Maureen Dowd's "Now there’s a gender gap in which men feel better about their lives". But those statements were false to start with, and not mainly because men were already self-reporting as tiny bit less likely to be "very happy", or a tiny bit more likely to be "not too happy". Those statements were false because they turn small statistical differences between very similar distributions into generic statements that readers interpret as describing characteristic properties of a group as a whole.

    Your correction nudges those small statistical differences a couple of percent further away from what Douthat and Dowd asserted. But the problem with their statements — which were fundamentally irresponsible — is not one that could be caused or cured by a statistical nudge in either direction.]

  10. blah said,

    September 21, 2009 @ 12:29 am

    One could also conclude that women are now less likely to lie about how happy they are than they were 40 years ago. The thumb suckers seem to make the unwarranted assumption that the self-reported state has always correlated exactly to the actual state, across time and across genders.

    [(myl) Let's call it "self-presentation" rather than "lying", and it's clear that there's an issue here. In a somewhat different area, namely sex differences in so-called "prosocial behavior", it's generally known (e.g. Richard A. Fabes and Nancy Eisenberg, "Meta-Analyses of Age and Sex Differences in Children's and Adolescents' Prosocial Behavior", 1998) that

    Sex differences were greatest when demand characteristics were high (i.e., it was clear what was being assessed) and individuals had conscious control over their responses (i.e., self-report indices were used); gender differences were virtually nonexistent when demand characteristics were subtle and study participants were unlikely to exercise much conscious control over their responding (i.e., physiological indices). Thus, when gender-related stereotypes are activated and people can easily control their responses, they may try to project a socially desirable image to others or to themselves.

    (More discussion of this point here.) It would be foolish to suppose that (changes in) sex differences in norms of self-presentation play no role in sex differences in self-reported "happiness". That said, the differences and the changes are in any case too small to motivate the use of generic plurals to make claims about how "men" and "women" feel.]

  11. LadyProf said,

    September 21, 2009 @ 1:55 am

    Bloix, I just want to thank you for your comment, which I saw in its earlier incarnation too. I assume (or hope) Maureen Dowd is kicking herself. The MSM and blogosphere haven't expressed enough gratitude: your point is devastating, and the silence in response tells us all we need to know about the purpose and uses of self-reported happiness data.

    [(myl) Is there any evidence that Maureen Dowd has ever regretted anything that she wrote, for any reason at all, much less because it merely turned out to be untrue or exaggerated? In this particular case, her 9/19 column is at the top of the NYT's "Most Emailed" list, and I'll bet that "kicking herself" is not the relevant reaction. ]

  12. Alas, a blog » Blog Archive » Link Farm and Open Thread, Brain Scanning Dead Fish Edition said,

    September 21, 2009 @ 3:39 am

    […] Once again, pundits are claiming that women's happiness has plummeted. And once again, Language Log is pointing out that the statistics don't support that claim. […]

  13. Bloix said,

    September 21, 2009 @ 11:19 am

    Blah – not to toot my own horn – well, yes, to toot my own horn, because if I don't who will, and anyway it's only a modest little tootle – you might take a look at what I and others said about possible changes in self-reporting of happiness among women over the last generation or so –
    http://languagelog.ldc.upenn.edu/nll/?p=1456

  14. Bloix said,

    September 21, 2009 @ 1:07 pm

    And to get back to the main point-
    Mark, if I'm conceptualizing your point correctly, as a complete lay person, what I'm understanding is something like this:

    We have, in effect, a three-dimensional data set – one dimension is number of persons, the second is degree of happiness ,and the third is time. That is, for each moment in time, we have two overapping two-dimensional curves – they may or may not be normal distributions – one for men, and one for women – and these curves represent the distribution of happiness in the population.

    The curves persist, with changes, over time, so they can be projected out longitudinally, as if they came forward toward you off the page. The proper graphical expression, then, would be two overlapping curved surfaces in a 3D space. If you had that 3D representation in front of you, you would see immediately that the vast majority of people, both men and women, fall under the sheltering space provided by both curved surfaces, so to compare the happiness of "men" to that of "women" makes no sense at all.

    But when the data is compressed into a two-dimensional space, so that you have mere lines for men and women, you lose the ability to see that most men and women are within the same general area. Thus the most salient information presented in the data is lost, and you are led to draw incorrect conclusions.

    Is that right?

  15. fish_bulb said,

    September 21, 2009 @ 2:05 pm

    Buckingham and other experts dispute the idea that the variance in happiness is caused by women carrying a bigger burden of work at home, the “second shift.” They say that while women still do more cooking, cleaning and child-caring, the trend lines are moving toward more parity, which should make them less stressed.

    If the amount of work outside the home is more than the decrease in the second shift, there would still be a net increase in work/stress.

    Of course this trend is stated with no evidence to back it up, or even 'a majority of the experts' explanation.

  16. MIOnline said,

    September 21, 2009 @ 3:37 pm

    One thing I noticed is that the points where people appear to be least happy coincided with recessions (e.g. 1980-82, 1990-92 and 2000-02) or other negative events (e.g. 9/11/01). Being unemployed or having your country in crisis would have to increase unhappiness in general.

    As for the purported decrease in happiness among women, perhaps increased anger (due to perceived discrimination) is to blame, while paradoxically women's lives have improved by many measures.

    It's also paradoxical that this so-called decrease in women's happiness would be going on when the suicide rate among women has decreased dramatically over the past few decades, while that of men has not.

  17. nascardaughter said,

    September 21, 2009 @ 6:03 pm

    I notice that Prof. Wolfers has a list of press reactions to "The Paradox of Declining Female Happiness" article on his UPenn page.

    What I wonder is, has he responded to any of them?

    [(myl) This list of points and counterpoints from 2007, originally given (and discussed) here, includes two blog posts by Justin Wolfers at Marginal Revolution.

    8/23/2007: Alan B. Krueger, "Are We Having More Fun Yet? Categorizing and Evaluating Changes in Time Allocation", ms. (Princeton U. and NBER).
    9/16/2007: Betsey Stevenson and Justin Wolfers, "The Paradox of Declining Female Happiness", ms. (University of Pennsylvania).
    9/26/2007: David Leonhardt, "He's Happier, She's Less So", NYT.
    9/26/2007: Jezebel, "Women Less Happy Than Men About Performing Every Single One Of Those Multi Tasks".
    9/26/2007: digg.com, "Men Are Now Happier Than Women".
    9/26/2007: Mark Liberman, "The 'happiness gap' and the rhetoric of of statistics", Language Log.
    9/27/2007: Mark Liberman, "Gender-role resentment and rorschach-blot news reporting", Language Log.
    10/1/2007: Steven D. Levitt, "Why Are Women So Unhappy?", Freakonomics Blog (NYT).
    10/1/2007: Mark Liberman, "Why are economists so misleading?", Language Log.
    10/2/2007: Jill Filipovic, "Feminists made their bed, now they have to lie in it alone with their cats", Feministe.
    10/2/2007: Amanda Marcotte, "Women: Not really that unhappy avoiding scowling cretins and petting cats", Pandagon.
    10/2/2007: Justin Wolfers, "The Significance of Changes in the Gender Happiness Gap", Marginal Revolution.
    10/3/2007: Echidne, "The Gender Happiness Gap", Echidne of the Snakes.
    10/2/2007: Jezebel, "Women Have Gotten Less happy, I'll Take My Graphing Calculator Out And Prove It".
    10/3/2007: Justin Wolfers, "The Real Significance of Changes in the Gender Happiness Gap", Marginal Revolution.
    10/3/2007: Steven D. Levitt, "The Debate on Female Happiness Heats Up", Freakonomics.

    I haven't looked for more recent responses.]

  18. Marie said,

    September 21, 2009 @ 6:31 pm

    Well, I have been happier since I started reading your analyses of all the gender difference claims out there. I think you do a great public service. And I keep what I've learned here in mind when I'm reading all sorts of other "statistically based news."

  19. Articles on women’s unhappiness make me unhappy. - The Pursuit of Harpyness said,

    September 22, 2009 @ 9:01 am

    […] paradox that may not even exist. Mark Liberman at the Language Log blog organized the General Social Survey results into a few graphs and charts that expose the empirical […]

  20. nascardaughter said,

    September 22, 2009 @ 9:10 pm

    Thanks for the links!

  21. Help On The Way For Unhappy Women « Happily Bitter said,

    September 24, 2009 @ 5:33 am

    […] Language Log […]

  22. UFO » Blog Archive » Sind Frauen wirklich unglücklicher? said,

    September 28, 2009 @ 3:18 pm

    […] Wie berichtet: Nein. […]

  23. Steve Fischer said,

    November 9, 2009 @ 1:38 pm

    In my opinion, these polls do a better job at gauging the fashionability of happiness than the prevalence of happiness. At times, happiness is "in," and at other times, there's nothing so chic as melancholy.

  24. Mädchenmannschaft » Blog Archive » Science FTW said,

    August 16, 2010 @ 5:37 am

    […] Heute ist der Effekt denn auch verschwunden. Und dass das „Happiness Gap” vor allem auf der großzügigen Interpretation minimaler statistischer Schwankungen beruht, kann nicht oft genug betont […]

  25. Rockin’ Poncho » Blog Archive » Facebook and sadness said,

    January 29, 2011 @ 11:33 pm

    […] than men because women are presented with so many more conflicting expectations. On the other hand here's a pretty good overview of skepticism about surveys that claim that "women are getting […]

  26. Frauen sind sogar noch glücklicher als Männer! « Drop the thought said,

    February 27, 2011 @ 5:17 pm

    […] habe ich das bei der Mädchenmannschaft damals nur am Rand erwähnt, aber wie Mark Libermann schon mehrfach schrieb, haben seine beiden Kollegen Betsey Stevenson und Justin Wolfers ihre Daten […]

RSS feed for comments on this post