Language Log

I very much enjoyed Andrew Gelman's post "Bayesian statistical pragmatism" (4/15/2011) on his blog Statistical Modeling, Causal Inference, and Social Science. And one aspect of that post struck me as especially relevant to some recent LL discussions:

I am surprised to see Kass write that scientists believe that the theoretical and real worlds are aligned. It is from acknowledging the discrepancies between these worlds that we can (a) feel free to make assumptions without being paralyzed by fear of making mistakes, and (b) feel free to check the fit of our models (those hypothesis tests again! Although I prefer graphical model checks, supplanted by p-values as necessary). All models are false, etc.

I assume that Kass is using the word "aligned" in a loose sense, to imply that scientists believe that their models are appropriate to reality even if not fully correct. But I would not even want to go that far. Often in my own applied work I have used models that have clear flaws, models that are at best "phenomenological" in the sense of fitting the data rather than corresponding to underlying processes of interest–and often such models don't fit the data so well either. But these models can still be useful: they are still a part of statistics and even a part of science (to the extent that science includes data collection and description as well as deep theories).

If you followed my advice (in "Norvig channels Shannon contra Chomsky", 5/31/2011) and read all of Peter Norvig's essay "On Chomsky and the Two Cultures of Statistical Learning", you may have detected a certain restrained testiness in Norvig's response. The goal of this post is to give a bit of explanatory background, and to suggest why, on the whole,  I share Norvig's reaction.

Several readers have sent in links to recent media coverage of C. Nathan DeWall et al., "Tuning in to psychological change: Linguistic markers of psychological traits and emotions over time in popular U.S. song lyrics", Psychology of Aesthetics, Creativity, and the Arts, 3/21/2011. For example, there's John Tierney, "A Generation’s Vanity, Heard Through Lyrics", NYT 4/25/2011.

[A]fter a computer analysis of three decades of hit songs, Dr. DeWall and other psychologists report finding what they were looking for: a statistically significant trend toward narcissism and hostility in popular music. As they hypothesized, the words “I” and “me” appear more frequently along with anger-related words, while there’s been a corresponding decline in “we” and “us” and the expression of positive emotions.

The latest xkcd:

The mouseover title: "Fun fact: if you say this every time a professor does something to a complex-number equation that drops the imaginary part, they'll eventually move the class to another room and tell everyone else except you."

It is well known that the same organism may be known by different common names in different areas (e.g. "cougar", "panther", "puma", and "mountain lion") and that the same common name may be used for different organisms in different areas (e.g. "blackberry"), but the assumption is that (pseudo-)Latin scientific names like Achillea millefolium "yarrow" are unique. Recent work by Kew Gardens and the Missouri Botanical Garden, with numerous collaborators, has revealed that this is not quite true: of 1.04 million species-level names, they classified only about 300,00 (29%) as accepted names. They classified 480,000 names (46%) as synonyms for accepted names and 260,000 (25%) as unresolved, meaning that the available data is not sufficient to determine whether or not they designate distinct species. By way of example, a query for Achillea millefolium reveals that it has synonyms such as Achillea ambigua, Achillea angustissima, Achillea borealis, and even some in other genera, such as Chamaemelum tanacetifolium. You can look things up yourself at The Plant List.

No word yet on beetles.

In "Snowclones are the dark matter of journalism", 1/24/2004, I noted the spread of the phrasal template X is the dark matter of Y: "The PC is the Dark Matter of the Internet", "Global technoscience is the dark matter of social theory", "Networking is the dark matter of high-speed internet", "Terrorism is the dark matter of the civilized world", "The extraterrestrial hypothesis is the dark matter of political science and science policy in the second half of the twentieth century", "Euroscepticism is the 'dark matter' of German politics", "the Boswell Co. now stands revealed for what it is: the dark matter of 20th century California history", "Intellectual property is the “dark matter” of the corporate universe".

A search today, almost seven years later, would turn up many more: "Untested code is the dark matter of software", "Influence is the Dark Matter of the Social Media Universe", "Organized crime is the dark matter of Ohio politics", and so on. And just this morning, I learned that dark matter has at least one scientific sense outside of physics.

Yesterday, the "most viewed" and "most emailed" item on the New York Times website was Deborah Tannen's essay, "Why Sisterly Chats Make People Happier", which opens this way:

"Having a Sister Makes You Happier": that was the headline on a recent article about a study finding that adolescents who have a sister are less likely to report such feelings as "I am unhappy, sad or depressed" and "I feel like no one loves me."

These findings are no fluke; other studies have come to similar conclusions. But why would having a sister make you happier?

The usual answer — that girls and women are more likely than boys and men to talk about emotions — is somehow unsatisfying, especially to a researcher like me. Much of my work over the years has developed the premise that women's styles of friendship and conversation aren't inherently better than men's, simply different.

A few days ago, I asked for help in tracking down some of the scientific support for Matt Richtel's claims about the bad effects of "digital overload" ("More factoid tracking", 9/1/2009). One of the more trackable factoids was the "study conducted at Stanford University, which showed that heavy multimedia users have trouble filtering out irrelevant information — and trouble focusing on tasks". And sure enough, The Neurocritic quickly came up with a reference that fits: Eyal Ophir, Clifford Nass, and Anthony Wagner, "Cognitive control in media multitaskers", PNAS, Published online before print 8/24/2009.

I believe that the full paper is freely available at the link given above (please let me know if this is wrong), and if you're interested in this topic, I urge you to read it. As in the case of the last paper by a Stanford psychologist that was discussed here, you should start by asking "Never mind the conclusions, what's the evidence?". And again, you may conclude that the descriptions of this research — in the popular press and even in the original paper — lead readers pretty far beyond the interpretations warranted by the research itself.

A month ago, I linked to Lera Boroditsky's WSJ piece "Lost in Translation", and promised to discuss the contents in more detail at some point in the future ("Boroditsky on Whorfian navigation and blame", 7/26/2010). At the time, I noted that there is probably no single linguistic idea that is more prone to exaggeration and mis-application than the "Sapir-Whorf hypothesis" about the relations between language and thought. And the WSJ editors' subhed for Boroditsky's article gives their readers a push down that road:

New cognitive research suggests that language profoundly influences the way people see the world; a different sense of blame in Japanese and Spanish.

Meanwhile, the NYT Sunday magazine has just published a major article by Guy Deutscher, "Does Your Language Shape How You Think?" (8/26/2010), which I hereby promise to discuss in detail at some point in the future. And in order not to let my neo-Whorfianism account fall too many promises in arrears, I'll actually post about Boroditsky's WSJ piece today. (I won't try to discuss both articles at the same time, because in this sort of thing, it's the scientific details that matter.)

There's new information emerging from the slow-motion Marc Hauser train wreck. Carolyn Johnson, "Journal editor questions Harvard researcher's data", Boston Globe 8/27/2010:

The editor of a scientific journal said today the only "plausible" conclusion he can draw, on the basis of access he has been given to an investigation of prominent Harvard psychology professor Marc Hauser's research, is that data were fabricated.

Gerry Altmann, the editor of the journal Cognition, which is retracting a 2002 article in which Hauser is the lead author, said that he had been given access to information from an internal Harvard investigation related to that paper. That investigation found that the paper reported data that was not present in the videotape record that researchers make of the experiment.

“The paper reports data … but there was no such data existing on the videotape. These data are depicted in the paper in a graph,” Altmann said. “The graph is effectively a fiction and the statistic that is supplied in the main text is effectively a fiction.”

Gerry Altmann posted a statement on his weblog with a more detailed account: harvard misconduct: setting the record straight", 8/27/2010). As indicated in Johnson's article, the facts and interpretations that Altmann provides go beyond, to a shocking degree, previously described issues of lost data or disagreement about subjective coding of animal behavior.

With a hat tip to Bruce Webster, more cartoons for the weekend, this time from Jonathan Rosenberg's Scenes from a Multiverse:

A reader asks the NYT Science Times (in C. Claiborne Ray's "Q+a" column on July 13):

Q. How does the weather service determine whether it was a tornado that caused the kind of destruction that recently occurred in Bridgeport, Conn.?

Before I go on, step back and ask yourself what sort of answer the reader was expecting.

I've been complaining for years about bad science writing in the popular press, and occasionally I've even made (futile) suggestions for improvement. This morning, though, I've realized that there's a cure.

But first, the disease.