Several readers have sent in links to recent media coverage of C. Nathan DeWall et al., "Tuning in to psychological change: Linguistic markers of psychological traits and emotions over time in popular U.S. song lyrics", Psychology of Aesthetics, Creativity, and the Arts, 3/21/2011. For example, there's John Tierney, "A Generation’s Vanity, Heard Through Lyrics", NYT 4/25/2011.
[A]fter a computer analysis of three decades of hit songs, Dr. DeWall and other psychologists report finding what they were looking for: a statistically significant trend toward narcissism and hostility in popular music. As they hypothesized, the words “I” and “me” appear more frequently along with anger-related words, while there’s been a corresponding decline in “we” and “us” and the expression of positive emotions.
Or David Brooks, "Songs about myself", NYT 4/13/2011:
This result isn’t surprising or controversial, but it’s nice to have somebody rigorously confirm an impression many of us have formed. In an upcoming essay for the journal, Psychology of Aesthetics, Creativity and the Arts, Nathan DeWall and others studied pop music lyrics between 1980 and 2007. They found that over these years lyrics that refer to self-focus and antisocial behavior have increased whereas words related to other-focus, social interactions and positive emotions have decreased. We’ve gone from “Love, Love, Love” to “F.U.”
Or "Study: Narcissism On Rise In Pop Lyrics", All Things Considered 4/26/2011:
So more than two decades ago, we were holding hands and swaying to the song of unity, and these days, we're bouncing to pop stars singing about how fabulous they are.
Psychologist Nathan DeWall has had the pleasure of listening to it all for research, and he found that lyrics in pop music from 1980 to 2007 reflect increasing narcissism in society.
Or Sandy Hingston, "Are today's kids narcissists?", Philadephia Magazine 4/27/2011:
Is there any hope for society amidst this onslaught of egotism? Jean M. Twenge, a co-author of DeWall’s study, has one suggestion for young people: “Ask yourself, ‘How would I look at this situation if it wasn’t about me?’” Unfortunately, that’s exactly the ability their overabundance of narcissism is unlikely to promote.
In "Lyrical narcissism?", 4/9/2011, I looked a little more closely at some of the data from this paper. See the original post for some additional details, but here's the authors' graph of (a parametric reconstruction of) their first-person singular counts:
And here's Cosma Shalizi's graph of the actual corresponding data, with a spline fit, 95% confidence intervals — and a convenient horizontal line, which falls within the confidence interval in 27 out of 28 cases:
I'm going to resist the temptation to analyze the rhetoric of the media presentations of this work in greater detail, or to relate this episode to the Great Happiness Gap of 2009. Instead, I'm going to offer some links and quotes from two relevant sources.
First, Ben Goldacre, "I foresee that nobody will do anything about this problem", Bad Science 4/23/2011:
Last year a mainstream psychology researcher called Daryl Bem published a competent academic paper, in a well respected journal, showing evidence of precognition. [...]
Now the study has been replicated. Three academics [...] have re-run three of these [...] experiments, just as Bem ran them, and found no evidence of precognition. They submitted their negative results to the Journal of Personality and Social Psychology, which published Bem’s paper last year, and the journal rejected their paper out of hand. We never, they explained, publish studies that replicate other work. [...]
We exist in a blizzard of information, and stuff goes missing [...]
The same can be said for the New York Times, who ran a nice long piece on the original precognition finding, New Scientist who covered it twice, the Guardian who joined in online, the Telegraph who wrote about it three times over, New York Magazine, and so on. [...]
What’s interesting is that the information architectures of medicine, academia and popular culture are all broken in the exact same way.
And second, Andrew Gelman, "Of beauty, sex, and power: Statistical challenges in estimating small effects", 9/10/2010, discusses "results [that] are more 'vampirical' than 'empirical' — unable to be killed by mere evidence".
He dissects a specific case (S. Kanazawa, "Beautiful parents have more daughters: a further implication of the generalized Trivers-Willard hypothesis", Journal of Theoretical Biology 2007) which yielded an "estimated effect of 4.7 percentage points (with standard error of 4.3)" such that "95% confidence interval is [-4%,13%]", from which Andrew concludes "Given that true effect is most likely below 1%, the study provides essentially no information". Nevertheless, through a fascinating chain of serial exaggerations, this work was featured by Steven Dubner in the Freakonomics blog as follows (emphasis added):
A new study by Satoshi Kanazawa, an evolutionary psychologist at the London School of Economics, suggests . . there are more beautiful women in the world than there are handsome men. Why? Kanazawa argues it's because good-looking parents are 36 percent more likely to have a baby daughter as their first child than a baby son — which suggests, evolutionarily speaking, that beauty is a trait more valuable for women than for men.
Andrew observes that
When using small samples to study small effects, any statistically significant finding is necessarily a huge overestimate.
And he notes that there are "incentives (in science and the media) to report dramatic claims".
My reluctant conclusion: if you read or hear about a scientific result in the mainstream media, the odds are depressingly good that it's nonsense.
Note — Andrew Gelman attributes the lovely "vampirical" coinage to Freese 2007, which appears to be Jeremy Freese, "The problem of predictive promiscuity in deductive applications of evolutionary reasoning to intergenerational transfers: Three cautionary tales", in Alan Booth et al., Eds., Intergenerational Caregiving, 2008.
Writing about the (ungeneralized) Trivers-Willard hypothesis, Freese observes that
Part of what makes the Trivers-Willard hypothesis perhaps more “vampirical” than “empirical”—unable to be killed by mere evidence—is that the hypothesis seems so logically compelling that it becomes easy to presume that it must be true, and to presume that the natural science literature on the hypothesis is an unproblematic avalanche of supporting findings. [...]
In fact, the Trivers-Willard hypothesis of adaptive sex ratio variation is not at all well established in the animal kingdom. Adaptive sex ratio variation in birds—including but not limited to hypothesis that one could characterize as extensions of the logic of the TWH (see Frank 1990 for a review)—is discussed at length in a meta-analytic study as an example in which publication bias can lead to distorted conclusions (Palmer 2000). That author finds that “on closer inspection few, if any, compelling data exist for adaptive departure from a 50:50 sex ratio in any species” [...]
The reason that animal results are relevant is that one can easily get the opinion that the TWH is a well-established phenomenon in animals and only through some pseudo-dualistic “human exceptionalism” might one resist its applicability to humans. Instead, we face the opposite of the famous line from “New York, New York”: if the hypothesis can’t make it there, should we expect it to make it anywhere?