Sax Q & A
« previous post | next post »
As a result of some Language Log posts a couple of years ago, I get quite a few inquiries from journalists about Dr. Leonard Sax and his science-based arguments for single-sex education. It's in the nature of things that only a small fraction of such discussions wind up in the resulting articles. For example, for Elizabeth Weil's NYT Magazine piece ("Teaching Boys and Girls Separately", 3/2/2008), I wound up sending about 4,000 words worth of emails to the author and her fact-checker, in response to their questions about specific points raised in some of Dr. Sax's writings. In the final article, this all wound up as background to a 250-word passage about sex differences in hearing. (See "Scupulously avoiding sigma", 3/2/2008, for some comments about other aspects of the article.)
I'm not complaining; Ms. Weil had a lot of material to cover, and she didn't have a lot of space to work with. However, another recent journalist's inquiry, raising some of the same issues, inspired me start a new policy. From now on, when I get inquiries from journalists, I'll try to post an edited version of my responses on Language Log. This may be of interest to some readers — and of course our famous money-back guarantee is available to the rest of you — and it will also make it easier for me to deal with subsequent questions about the same issues.
In this case, I'll start with my responses to the four new questions that arrived yesterday afternoon. The answer to the last one brings up some of those emails sent to Elizabeth Weil, which I'll post in an edited form later this weekend.
1. I've read a few posts on Language Log, but please tell me more about what you think about Dr. Sax's arguments about sex-based differences in the brain?
In his books, Leonard Sax is a political activist using science to make a case, not a scientist evaluating a hypothesis.
Science is sometimes on his side, sometimes neutral or equivocal, and sometimes against him. He picks the results that fit his agenda, ignoring those that don't; and all too often, he misunderstands, exaggerates or misrepresents the results that he presents.
There's detailed support for these assertions in some Language Log posts from 2006:
"David Brooks, cognitive neuroscientist" (6/12/2006)
"Are men emotional children?" (6/24/2005)
"Of rats and (wo)men" (8/19/2006)
"Leonard Sax on hearing" (8/22/2006)
"More on rats and men and women" (8/22/2006)
"The emerging science of gendered yelling" (9/5/2006)
"Girls and boys and classroom noise" (9/9/2006)
This doesn't mean that his conclusions are false, but it does mean that his appeals to science are not trustworthy.
2. Speaking as a linguistics professor, why do you think girls have better verbal skills than boys?
This question assumes that girls have better verbal skills than boys. In fact, that generalization is somewhere between misleading and false. And you don't have to be a linguistics professor to figure this out — you just need to look at the published numbers from tests related to verbal skills, with a little bit of basic statistics to see what they mean.
I encourage you to look into it for yourself — some references are below — but here's a summary of what (I think) it all means in practical terms.
If you pick a hundred girls and a hundred boys at random, and give them a battery of standardized tests measuring various sorts of verbal abilities, and split the group into upper and lower halves based on the results, the highest-scoring hundred kids will probably include about 52 girls and 48 boys, while the lower-scoring hundred kids will be about 48 girls and 52 boys. (This is based on the 0.10 effect size that is calculated for studies since 1973 in the Shibley & Linn meta-analysis, cited below. The details will vary quite a bit, depending on what tests you use, and what population of girls and boys you sample from — for some tests and some populations, the top half is likely to have more boys in it than girls.)
This is a difference, but it's not a very big one. The effects are small, complex, and variable over time and social setting. In my opinion, the causes remain uncertain.
According to Janet Shibley Hyde and Marcia C. Linn, "Gender Differences in Verbal Ability: A Meta-Analysis", Psychological Bulletin, 104:1 53-69 (1988):
Many regard gender differences in verbal ability to be one of the well-established findings in psychology. To reassess this belief, we located 165 studies that reported data on gender differences in verbal ability. The weighted mean effect size (d) was +0.11, indicating a slight female superiority in performance. The difference is so small that we argue that gender differences in verbal ability no longer exist. Analyses of effect sizes for different measures of verbal ability showed almost all to be small in magnitude: for vocabulary, d = 0.02; for analogies, d = −0.16 (slight male superiority in performance); for reading comprehension, d = 0.03; for speech production, d = 0.33 (the largest effect size); for essay writing, d = 0.09; for anagrams, d = 0.22; and for tests of general verbal ability, d = 0.20. For the 1985 administration of the Scholastic Aptitude Test-Verbal, d = −0.11, indicating superior male performance. Analysis of tests requiring different cognitive processes involved in verbal ability yielded no evidence of substantial gender differences in any aspect of processing. Similarly, an analysis by age indicated no striking changes in the magnitude of gender differences at different ages, countering Maccoby and Jacklin's (1974) conclusion that gender differences in verbal ability emerge around age 11. For studies published in 1973 or earlier, d = 0.23 and for studies published after 1973, d = 0.10, indicating a slight decline in the magnitude of the gender difference in recent years.
[At this point, the reader obviously needs to understand what "effect size" means. It's a measure of between-group differences, expressed in terms of the amount of within-group variation; and the usual qualitative interpretation is that 0.2 is a small effect size, 0.5 is a moderate effect size, and 0.8 or larger is a large effect size. In more technical language, this is "Cohen's d", the difference in means divided by the pooled standard deviation. As a point of comparison, the effect size for sex differences in height among American 18-year-olds is about d=2.21. For more discussion of the concept, try my post "Gabby guys: the effect size" (9/23/2006) or the Wikipedia article. For a discussion of why it matters that most citizens (even intellectuals) have no clue about any of this, see "The Pirahã and us" (10/6/2007).]
A more recent meta-analysis gave a table of 124 quantitative estimates of sex-difference effect sizes, taken from 46 published meta-analyses of gender differences in general (Janet Shibley Hyde, "The Gender Similarities Hypothesis", American Psychologist, 60(6): 581-592, 2005) . These deal with many different cognitive and behavioral measures, from "mathematics computation" to "job attribute preference". Pulling out those that compared verbal skills, we get:
| Study and variable | Age | No. of reports | Effect size (d) |
| Hedges & Nowell [1995] | |||
| Reading comprehension | Adolescents | 5* | +0.09 |
| Vocabulary | Adolescents | 4* | -0.06 |
| Feingold [1988] | |||
| DAT spelling | Adolescents | 5* | +0.45 |
| DAT language | Adolescents | 5* | +0.40 |
| DAT verbal reasoning | Adolescents | 5* | +0.02 |
| Hyde & Linn [1988] | |||
| Vocabulary | All | 40 | +0.02 |
| Reading comprehension | All | 18 | +0.03 |
| Speech production | All | 12 | +0.33 |
The asterisks indicate that "data were from major, large national samples". Positive values of d represent higher scores for females, negative values indicate higher scores for males (I've switched the sign compared to Table 1 in the source, in order to make the sign consistent with the practice in Hyde & Linn above.). Although 7 of the 8 meta-analyses show an advantage for females, only three of those are non-negligible in magnitude.
And the biggest effect sizes — 0.45 and 0.40 from the Feingold [1988] study — are worth looking into a little further. That study is Alan Feingold, "Cognitive Gender Differences Are Disappearing", American Psychologist 43(2) 95:103, 1988. Its abstract:
Gender differences in cognitive abilities were determined using the norms from the four standardizations of the Differential Aptitude Tests conducted between 1947 and 1980, and from the four standardizations of the Preliminary Scholastic Aptitude Test/Scholastic Aptitude Test conducted between 1960 and 1983. The standardized gender differences (ds) were averaged over grade of examinees and year of standardization to obtain a mean effect size for each ability, and variations among effect sizes were examined for grade, year, and Grade × Year trends. Girls scored higher than boys on scales of grammar, spelling, and perceptual speed; boys had higher means on measures of spatial visualization, high school mathematics, and mechanical aptitude; and no average gender differences were found on tests of verbal reasoning, arithmetic, and figural reasoning. Gender differences declined precipitously over the years surveyed, and the increases in these differences over the high school grades have diminished. The important exception to the rule of vanishing gender differences is that the well-documented gender gap at the upper levels of performance on high school mathematics has remained constant over the past 27 years.
Here are some relevant numbers from Feingold's paper, dealing with SAT verbal scores from 1960 to 1983. Again, the values are effect sizes, with positive values indicating that girls' average scores were higher, and negative values indicating that boys' average scores were higher. The most important thing is that all the effect sizes are small; but it's also interesting to look at the trend:
| Year | Juniors | Seniors |
| 1960 | NA | +0.06 |
| 1967 | +0.05 | +0.01 |
| 1974 | +0.02 | -0.04 |
| 1983 | -0.11 | -0.08 |
Just to show how different things can be, for a different test in a different language and society, here are some numbers from Robert Colom et al., "Are cognitive sex differences disappearing? Evidence from Spanish populations", Personality and Individual Differences 27(6): 1189-1195, 1999.
This is from "the Spanish translation of the DAT", which is "a battery of six reliable group-administered paper-and-pencil tests, which measure "verbal ability (Verbal Reasoning, VR), perceptual speed (Clerical Speed and Accuracy, CSA), three dimensional spatial visualization (Space Relations, SR), artihmetic (Numerical Ability, NA), mechanical ability (Mechanical Reasoning, MR), and figural reasoning (Abstract Reasoning, AR)". The American data from the DAT Verbal Reasoning subtest yielded an effect size of 0.02 in favor of females, but the Spanish data from the translation of the same test showed an effect size of 0.30 in favor of males in 1979 and 0.31 in favor of males in 1995. Since these are large national studies in both cases, the source of the difference is certainly not statistical sampling error. And I very much doubt that there can be significant sex-linked population-genetic differences between the U.S. and Spain. There must be some differences in the test-taking population, or the culture, or the school system.
There are many interesting scientific questions about where measured sex differences (and other group differences) in verbal abilities come from. (For a review as of a decade ago, see Alan Feingold, "Cognitive gender differences: Where are they, and why are they?", Learning and Individual Differences 8(1):25-32, 1996.) But in my opinion, such differences are nowhere near large enough or consistent enough to be an argument for sex-segregated education.
3. What do you think is the reason for the widening gender gap in American schools? Are single-sex schools the answer?
The short answer to both questions is that I don't know.
But my impression is that the main overall trend is for girls and women to do better rather than for boys and men to do worse; that this is a world-wide effect, which is unlikely to be the result of U.S.-specific educational or social developments over the past few decades.; and that the causes are almost certainly not sex differences in perceptual or cognitive skills or styles.
Educational trends in the Arab world provide a striking case study. In response to my discussion of David Brooks' channeling of Leonard Sax ("David Brooks, Cognitive Neuroscientist", 6/12/2006), Lameen Souag wrote
I wonder how Brooks would account for similar phenomena elsewhere, such as Qatar, where men's dropout rates are higher than women's even at primary school and more than twice as many women as men attend university, or Algeria, where 20% more women than men make it to the baccalaureate, or Kuwait, where two-thirds of university students are women. Learning styles yes – sitting down in one place and paying attention all day is a sore trial for most boys – but there's surely something broader going on here than choice of violence-filled vs. touchy-feely literature, never mind his further inferences about brains.
Qatar: http://lughat.blogspot.com/2006/04/more-from-qatar.html
Algeria: http://jazairana.blogspot.com/2006/06/60-of-bac-candidates-are-women.html
Kuwait: http://gender.pogar.org/countries/country.asp?cid=8
As far as I can tell, most if not all schools in Qatar continue to be segregated by sex (see this Rand report, which says that "Three levels of general education are provided: primary (grades 1-6), preparatory (grades 7-9), and secondary (grades 10-12), with girls and boys in separate schools.") I'm not sure whether Algerian schools are single-sex or not. The cited link suggests that university classes have been segregated by sex in Kuwait since 1996, and I presume that elementary and secondary school are sex-segregated there as well.
The educational "gender gap" is clearly a serious issue, one that deserves careful study and attention. Leonard Sax has argued forcefully that it results from large differences between the sexes in many perceptual, cognitive and behavioral dimensions: "Girls and boys play differently. They learn differently. They fight differently. They see the world differently. They hear differently." But I've argued that Dr. Sax's account of sex differences in sight and hearing, at least, is highly exaggerated.
And there are many studies suggesting that the most important factors in this area are not perceptual or cognitive at all. Thus Brian A. Jacob, "Where the boys aren't: non-cognitive skills, returns to school and the gender gap in higher education", Economics of Education Review 21(6) 589-598, 2002:
Nearly 60 percent of college students today are women. Using longitudinal data on a nationally representative cohort of eighth grade students in 1988, I examine two potential explanations for the differential attendance rates of men and women—returns to schooling and non-cognitive skills. […] I find that higher non-cognitive skills and college premiums among women account for nearly 90 percent of the gender gap in higher education. Interestingly, non-cognitive factors continue to influence college enrollment after controlling for high school achievement.
It's also important to be clear about where there are gaps and where there aren't, and what the trends really are. Thus Muna Husain and Daniel L. Millimet, "The Mythical 'Boy Crisis'?", Economics of Education Review, in press, 2008:
The popular press has put forth the idea that the US educational system is experiencing a “Boy Crisis,” where boys are losing ground to girls across multiple dimensions. Here, we analyze these claims in the context of math and reading achievement during early primary school. We reach two conclusions. First, white boys outperform white girls in math across virtually the entire distribution by the end of third grade; there is less evidence for other races. Second, boys lag behind girls in reading at the start of kindergarten and at the end of third grade across all races, but only the lowest-achieving boys lose ground over the first four years; boys gain ground between first and third grades.
4. Dr. Sax posted on his Web site that he has sent you rebuttals to your Language Log posts but you have yet to respond. He has also posted these letters on his Web site. Would you like to address this?
From the page "About Leonard Sax MD PhD" at the web site of the National Association for Single Sex Public Education:
Mark Liberman, at the University of Pennsylvania, has posted several blogs attacking Dr. Sax's positions regarding sex differences in hearing and vision. Dr. Sax has replied directly to Professor Liberman (via snail mail). Having received no response from Professor Liberman, Dr. Sax has agreed to post these letters online.
Click here to read Dr. Sax's letter regarding sex differences in hearing;
click here to read Dr. Sax's letter regarding sex differences in vision.
My blog posts from 2006 criticized Dr. Sax for exaggerating, misunderstanding or misrepresenting sex differences described in scientific studies of hearing and vision. In these letters, originally sent to me a few months ago in paper form, he counters mainly by citing other studies.
This doesn't affect my original complaint of exaggeration, misunderstanding and misrepresentation. And I've been hoping to avoid a protracted controversy about points that I don't believe are very much in doubt. However, I've read the additional references that he brings forward in these letters — that's mostly what those 4,000 words of email to Elizabeth Weil and her fact checker were about — and I'll post something about them in a few days.
[Update — these posts are "Liberman on Sax on Liberman on Sax on hearing", 5/19/2008; and "Retinal sex and sexual rhetoric", 5/20/2008.]
[I've left comments off for this post. However, feel free to send me email on the subject, and I'll add updates or corrections as appropriate. And after I've posted my response to Dr. Sax's letters, I'll set up an open thread on the subject for those who want to register their opinions, suggestions, anecdotes, and so forth.]
[Now that her series has been published, I'll reveal that the journalist who asked these questions was Amanda Schaffer.]