There's been a lot of discussion, both in the mainstream media and on the intertubes, of a study that came out a couple of days ago in the Proceedings of the National Academy of Sciences: Ivanka Savic and Per Lindström, "PET and MRI show differences in cerebral asymmetry and functional connectivity between homo- and heterosexual subjects", PNAS, 6/16/2008.
And in this case, the study's rhetorical reception is more interesting than the study itself.
Let's start with a minor point of interest: the way that MSM editors chose to spin their headlines. Like the study's title in PNAS, the headline of the PNAS press release treated the four categories in the study equivalently: "Symmetry of Homosexual Brain Resembles That Of Opposite Sex, Swedish Study Finds". And a few news outlets took this same path: "Gay Brain Structure Similar to Straight Opposite Sex", Bloomberg; "Homosexual brain resembles that of opposite sex", AFP; "Scientists Link Brain Symmetry, Sexual Orientation", Wired News; "Gay brains structured like those of the opposite sex", New Scientist.
However, the great majority of headline writers took another approach.
Others — the largest group — laid it out in terms of the similarity of homosexual men and heterosexual women: "Gay men and heterosexual women have similarly shaped brains, research shows", The Guardian; "Gay men and straight women share brain detail", Reuters; "Study: Gay Men, Straight Women Share Brain Characteristics", Fox News; "Gay men and heterosexual women have similar brains, study finds", The Telegraph; "Gay Men, Straight Women Have Similar Brains", Washington Post; "Scans find gays' feminine side is in the brain", Times Online; "Gay Men, Straight Women Have Similar Brains", National Geographic; "Why straight women and gay men get along so well", 680 News; "Gay Men, Straight Women Share Same Brain", eCanada Now; "Women 'share same emotional brain traits' as homosexual men", Daily Mail; "Gay men and straight women have similar brains", Salon.
I found only one headline that highlighted the similarity of homosexual women and heterosexual men, and this was a painful pun in the satirical IT rag, The Register: "Lesbians like straight men, researchers find". (Well, there was a blog post "Straight men and gay women have similar brains" by Daniel Cressey at Nature, making this same meta-analytic point about the distribution of headlines.)
There were no headlines of the form "What the straight brain looks like", or "Brains of heterosexuals are different, study finds".
Scientific American's headline was ambiguous, and false on both readings: "Study Says Brains of Gay Men and Women Are Similar". This might attribute similarity to "gay [men and women]", i.e. to gay men and gay women; or it it might be parsed as "[gay men] and women", i.e. gay men and women of whatever kind. Either way, it's wrong.
But much more interesting than the headlines, in my opinion, was the uniformly essentialist language in which all sources describe the study's results. In this respect, the journalists all follow the PNAS press release:
Ivanka Savic and Per Lindström, of the Department of Clinical Neuroscience at the Karolinska Institute in Stockholm, Sweden, now report that the brains of heterosexual men and homosexual women are slightly asymmetric—the right hemisphere is larger than the left—and the brains of gay men and straight women are not. [emphasis added]
It's natural to read this as a prediction about the brains of individual men and women. If you're a heterosexual male or a homosexual female, then your cerebral hemispheres will be slightly asymmetric. If you're a homosexual male or a heterosexual female, then your cerebral hemisphere will not have this property. Right? Nope, sorry, not even close.
Let's ignore the problematic assumption that all humans divide neatly into these four categories of sexual orientation, and assume that we're talking about people at the extreme ends of the Kinsey scale (though I don't think this is actually true of the study in question). [Added later: yes it was -- the HeM and HeW were 0, the HoM 6, and the HoW 5.5 on the Kinsey scale -- sorry, I missed that earlier.] What did the study actually find about brain symmetry? Here's the crucial table (click on the image for a larger version):
Among the 25 heterosexual males in the study, the average size of the right cerebral hemisphere was 624 cc, and the average size of the left cerebral hemisphere was 612 cc, or about 1.9% smaller. Among the 20 homosexual women in the study, the right hemisphere averaged 548 cc, while the left hemisphere averaged 543 cc, or about 1.5% smaller.
OK, "slightly asymmetric", we're with the program so far.
And the 25 heterosexual women averaged 581 cc on the right, and 581 on the left; while the 20 homosexual males averages 608 on the right and 609 on the left.
OK, check. Slight asymmetry for the straight males males and the lesbians, not for the gay guys and the straight women.
But wait a minute. There are those ± thingies, the "standard deviations". If the cerebral-hemisphere volumes are distributed "normally", as we would expect, and if the estimates of means and standard deviations in the table were actually the true population values values, then we can calculate what the expected distribution of hemispheric volumes (and the expected distribution of hemispheric asymmetries) will look like, for individuals within each sex-by-sexual orientation category. And when we do that, a completely different picture emerges.
For the heterosexual males in the study, the distribution of right and left hemispheric volumes is predicted to look like this:
And among the individuals in this group, the right hemisphere will be greater than the left hemisphere about 57.9% of the time, while the left hemisphere will be greater about 42.1% of the time. (Assuming independence — which is an inappropriate assumption, since the variables are probably correlated…) This predicts that among the 25 HeM subjects in the study, 14 should exhibit the "characteristic" right-greater-than left asymmetry, while 11 do not.
For the 25 HeW women, since their average values are equal, half will be predicted to be asymmetrical in one direction, and half in the other — this predicts that 12 or 13 should show the right-right-than-left asymmetry supposedly characteristic of heterosexual males, while 12 or 13 do not.
If we do the same calculations for the means and standard deviations reported for the other categories, we get predictions that might have been presented as follows:
Rightward hemispheric asymmetry was found in the brains of 14 of 25 heterosexual males and 11 of 20 homosexual females, but in only 13 of 25 heterosexual females and 10 of 20 homosexual males.
How much media play do you think the study would have gotten, if the results had been spun like that?
Or to put it another way, how many readers of the media descriptions do you think understood the story in those terms?
[added later: As Brett points out in a comment, the fact that the hemisphere sizes are correlated means that assuming independence decreases the effect. To allow for this, I should instead look at the by-category means and standard deviations of their "asymmetry index" AI = (R-L)/(R+L), and estimate how often this will be greater or less than 0. This predicts results like:
Rightward hemispheric asymmetry was found in the brains of 18 of 25 heterosexual males and 17 of 20 homosexual females, but in only 11 of 25 heterosexual females and 10 of 20 homosexual males.
If that's how it came out, it's a more striking result -- but still far from categorical...]
I'm not arguing that this paper's results are scientifically meaningless, just that they don't mean what nearly everyone reading the media coverage — and the PNAS press release — thinks they mean.
[While we're discussing the details of the study, I should point out one other possible issue. The authors tell us that "The inter-rater correlations was [sic] 0.85 … for cerebral hemispheres", which is not a wonderful number, given that we're looking for a one or two percent difference in samples of 20 or 25.]
The re-spun numbers I've just given are the likeliest outcomes given the cited means and standard deviations, but the actual counts were no doubt different, because with such small sample sizes, the outcomes for different samples are likely to vary quite a bit. The study's authors don't give us the actual numbers, but they do give some information to suggest that the situation is not quite as bad as my numbers suggest, though it is still a long way from the essentialist implications of their discussion:
A post hoc evaluation detected asymmetry exceeding two standard deviations of that in HeW (who were hypothesized to have symmetrical volumes) in 1 HeW, 4 HoM, 7 HoW, and 12 HeM.
I'm a bit puzzled about how 12 out of 25 HeM could wind up with asymmetries greater than 2 standard deviations, given a difference in means of 12 cc on standard deviations of 43 and 41, i.e. mean left and right hemispheres differing by about 0.28 standard deviations. Either there's something funny about the distribution, or I'm not understanding the statement correctly. But even on this way of presenting things, which I'm sure was selected to make the groups seem as different as possible, it's clear that we're looking at a graded set of differences, not a categorical distinction.
[added later: Oh, I think I see what they did. They're looking at the distributions of their "asymmetry index" (AI), (R-L)/(R+L), expressed in terms of the standard deviation of the AI for HeW, which was 0.005. Thus the average HeM AI of 0.012 ±0.02, which differs from 0 by 0.012/0.02 = 0.6 of a standard deviation in terms of its own standard deviation, differs from 0 by 0.012/0.005 = 2.4 if you measure it in terms of the variability in the HeW distribution. So if you look at the individual HeM AIs, it could well be true that almost half of them are more than two HeW standard deviations away from zero. If this is what they did, it's is tricky language at best, it seems to me...]
Where does this confusion come from? Well, there's statistical illiteracy on the part of flacks, journalists and the general public. And this ignorance is fertile ground for a certain kind of quasi-dishonest hucksterism on the part of the flacks (who are paid to make things seem more interesting), the journalists (who will get better placement for more interesting-seeming stories), and, alas, the scientists as well, who are also interested parties.
But from a more abstract perspective, perhaps the problem is a linguistic one. The modern English language lacks good resources for discussing sampled properties of groups, in the same way that the Pirahã language is inadequate for dealing with arithmetic (see "The Pirahã and us", 10/6/2007, for some additional discussion of this point).
You could argue that this is just a lexical and cultural problem. We already have words and phrases like "distribution" and "percentile" and "standard deviation" and "effect size"; maybe the problem is that most people don't understand these concepts, and even those that understand them don't bring them to mind when they should.
But I think it's not quite that simple. It's hard to banish plural nouns and other simple and natural ways of referring to groups, as in these quotations from the Savic and Lindström paper:
HeM and HoW showed a rightward cerebral asymmetry, whereas volumes of the cerebral hemispheres were symmetrical in HoM and HeW.
HeM and HoW had significantly asymmetrical hemispheric volumes … In contrast, no asymmetry was detected in HeW or in HoM.
And once statements of this kind are on the table, it's hard not to take them as shorthand for universally-quantified generalizations, or (more likely) assertions about group archetypes or prototypes.
I don't have a solution to propose. But it's clear to me that we have a problem.
[I should mention in passing that Savic and Lindström's results about PET-derived amygdala connectivity can't be evaluated in the way that I've evaluated their hemispheric asymmetry data, because they only present pooled group data, in which
Functional connectivity was defined as the extent to which normalized rCBF in seed VOIs covaried with pixel-based rCBF values across the investigated subjects. The normalized rCBF was extracted from circular (5-mm) VOIs covering the right and left amygdala ... Significant covariations ... were calculated by using the entire brain as search space (multisubject condition and covariate analysis within SPM2).
That is, they looked for covarying brain regions within all of the pooled data for heterosexual males, all of the pooled data for homosexual females, etc. As a result, I don't see any way to characterize the amount of individual variation within each sexual-orientation group, or to compare it to the differences between groups. Based on the nature of their experiments, and what I know about the amount of individual variation in such experiments, I suspect that if you tried classify individuals' sex and sexual orientation on the basis of this data, the results would not show any sort of crisp or categorical differentiation.
One other aspect of their study puzzled me. The most striking fact about the measured hemispheric volumes, it seems to me, were the differences between groups in overall brain size. Thus the total cerebral volumes were these:
The fact that the heterosexual men's brains (well, the cerebral cortex parts) were about 6.4% larger than the heterosexual women's brains is as expected. But the fact that the heterosexual women's brains were on average about 6.5% larger than the homosexual women's brains was not something that I expected. (And the effect size in this comparison is apparently about 1.7-1.9, guessing from the standard deviations of the hemispheric measurements, which is a pretty big effect.) The heterosexual men's brains were about 1.6% larger on average than the homosexual men's brains. (Here the effect size is probably only about 0.4 to 0.5.)
The authors don't comment on this -- maybe it's a well-known effect that I've missed, though I can't find it in a quick glance at the literature on the neurology of sexual orientation -- but I wonder about it. If it's real, it's a much larger effect than the differences in asymmetry, and it presumably means something important. If it's an artefact of some sort that doesn't replicate, then it calls other aspects of the study into question.]