Cultural diffusion and the Whorfian hypothesis

« previous post | next post »

Geoff Pullum summarizes Keith Chen's view of "The Effect of Language on Economic Behavior" as follows ("Keith Chen, Whorfian economist", 2/9/2012):

Chen [...] thinks that if your language has clear grammatical future tense marking [...], then you and your fellow native speakers have a dramatically increased likelihood of exhibiting high rates of obesity, smoking, drinking, debt, and poor pension provision. And conversely, if your language uses present-tense forms to express future time reference [...], you and your fellow speakers are strikingly more likely to have good financial planning for retirement and sensible health habits. It is as if grammatical marking of the difference between the present and the future insulates you from seeing that the two are coterminous so you should plan ahead. Using present-tense forms for future time reference, on the other hand, encourages you to see that the future is just more of the present, and thus encourages you to put money in a 401(k).

Geoff notes that "Chen's evidence on the lifestyle indicators comes from massive amounts of hard data, and his mathematical analysis is serious". But in addition to expressing some qualms about the linguistic data, Geoff worries that the large number of linguistic traits and the large number of lifestyle and other cultural traits might give rise to spurious connections:

I also worry that it is too easy to find correlations of this kind, and we don't have any idea just how easy until a concerted effort has been made to show that the spurious ones are not supportable. For example, if we took "has (vs. does not have) pharyngeal consonants", or "uses (vs. does not use) close front rounded vowels", would we find correlations there too?

I have similar concerns; but I believe that I can explain and justify my worries without looking at any real data at all. There are two qualitative facts about the world that make it especially easy to fool ourselves about quantitative connections of this kind.

The first relevant fact is that cultural traits — whether language-related or lifestyle-related — tend to diffuse geographically. As a result, there are fewer degrees of freedom in our data than we might think.

Here's a simple demonstration. Suppose we assign uniform random values on the open interval 0,1 to points on a 40×40 grid:

Now we do the same thing, independently, 50 times over. When we're done, each of the 1600 cells has 50 numerical traits or features.  We can check the correlations among the geographical distributions of these 50 traits or features, or look in various ways at the effectiveness of attempts to predict the distribution of one of them in terms of the distribution of another; and we'll find that our random number generator has done its job.

But suppose we implement a simple model of cultural diffusion, assuming that we have a series of time-steps on which for each feature, each cell independently influences its immediate neighbors, and is influenced by them in turn. The details don't really matter much — what I did is explained at the end of the post. The result, by design, is that each feature diffuses into a pattern of random regional blotches:

And even though each feature evolved completely independently of all the others, this blotchiness means that (positive or negative) geographical correlations between features become more common. (The plot below shows the distribution of all (50*50 – 50)/2 = 1225 pairwise correlations…)

As the blotches spread, the distribution of correlations also spreads out, and more significant-seeming correlations get commoner.  The emergent blotch-correlations of feature-distributions look like this:

Those pictures show the geographical distributions of the 16th feature and the 40th feature after 100 time steps on that particular run, and you can see that they are weakly negatively correlated (at around r = -0.25). If we apply logistic regression, as Chen did, we find that the beta parameter for predicting feature 16 in terms of feature 40 is -1.08, with an estimated standard error of 0.21, p<.0000001 (or corrected for multiple comparisons, p=.0011).

The plot below compares the predicted values (on the left) with the observed values (on the right):

But there's a second simple fact about the world that also tends to introduce artefacts into an analysis of this kind. Cultural traits not only spread geographically — they tend to spread in correlated ways.

In what we've done so far, each of the traits diffuses independently from all the others. There are no rivers or mountain ranges steering cultural diffusion along  preferred pathways. There's no long-distance migration or colonization. And we took no account of the fact that even in simple cases of local borrowing, cultural traits tend to travel in packages: if culture X influences your language, there's a good chance that it influences other aspects of your culture as well. Introducing such co-diffusion of traits into the model would tend to increase the similarity of feature-blotch distributions, and therefore would increase the number of correlations for which we might postulate a causal explanation.

But all this is entirely independent of any genuine causal connections among the traits. For example, if you borrow your writing system from the Chinese, it's likely that you'll borrow some aspects of food culture and architecture as well. This is not because a logographic writing system tends to cause people to use chopsticks or build pagodas.

My point is not that Prof. Chen's analysis is wrong. There may very well be consequential connections of the kind that he explores. But the existence of statistically significant logistic regression coefficients among geographic distributions of cultural traits is not enough to convince me.

In the simulation above, "neighbors" are defined by modular arithmetic on array indices, so that the top row is next to the bottom row, and the rightmost column is next to the leftmost one. At each time step, each cell interacts with each of its neighbors (and itself) by flipping an unfair coin for each trait, with probabilities determined by its current value for that trait; the neighbor then updates the trait in question as a linear combination of its old value and the 0 or 1 derived from the coin flip.

Corresponding Matlab code is here.



  1. Chris said,

    February 12, 2012 @ 5:15 pm

    Unfortunately, Andrew Sullivan has used his huge blog megaphone, which reaches millions, to disseminate Chen with zero critical analysis of its flaws:

  2. GeorgeW said,

    February 12, 2012 @ 6:44 pm

    If we would all just speak Gulf Arabic, the energy crisis would be solved.

  3. phspaelti said,

    February 12, 2012 @ 8:53 pm

    A question that I had was that in such research people are mapping linguistic features — which are typically categorical — onto features which are typically gradient, and in fact multi-dimensional. So "has future tense" is binary, while "plans for the future" has many dimensions all of which are highly gradient ("is obese", "saves money", "builds sturdily", etc.). If on top of this the categorical linguistic feature is misanalyzed, then one is mapping an inconsistent single variable onto a multicolored map. Obviously confirmation will be in the eye of the beholder.

    [(myl) The boolean nature of the independent variable isn't in itself a problem -- in principle a yes/no variable might have considerable predictive power. In a drug trial, for example, the independent variable is drug or no drug, and there may be several gradient dependent variables measuring the outcomes. Nor is noise necessarily a problem -- it just means you need more data to see the signal, as long as the noise is unbiased relative to the dependent variables of interest.

    But when the observations -- whether cardinal or ordinal or boolean -- come from a geographical pattern with areal spread, then it may be a bit like doing a drug trial where maybe 2/3 of the patients who got the drug also happen to be pregnant women, whereas half of those who didn't also happen to be men over the age of 70.]

  4. Chad Nilep said,

    February 12, 2012 @ 11:52 pm

    "But when the observations… come from a geographical pattern with areal spread, then it may be a bit like doing a drug trial where maybe 2/3 of the patients who got the drug also happen to be pregnant women, whereas half of those who didn't also happen to be men over the age of 70."

    I nominate this sentence for simile of the week. Matt Taibbi, eat your heart out.

  5. MattF said,

    February 13, 2012 @ 8:31 am

    Really interesting. I'll note that red correlation distribution curve suggests a test for significant deviation from randomness.

  6. Tom W said,

    February 13, 2012 @ 9:38 am

    Chen opens up fascinating insights into the language of squirrels, who we now know hide their acorns because they have no future tense! But even if Whorfianism is true, why would the linguistic structure that makes people provide for the future have to do with the future tense? What adult speaker of any language can't conceive of tomorrow or doesn't think that it will dawn? Wouldn't being provident more likely involve the conditional? It's more an ability to think in terms of evenualities, not futurities, that prompts one to provide for the future. Not that anyone with an understanding of the complexity of the brain, let alone the compexity of cultural diffusison that you explicate so well, should expect even that hypothesis to pan out.

  7. Glen Gordon said,

    February 13, 2012 @ 12:41 pm

    This sounds like an over-extension of the idea that language styles are influenced by personality styles. Not entirely crazy to me but certainly this connection can go overboard.

    It seems popular enough in the corporate world when we're advised to avoid passives lest we're mistaken by simplistic interviewers as passives in our daily lives. Hey, here's a thought. Maybe active people use passives too. I know! Crazy, right?? But hey who am I but an iconoclastic pleb to think such unpopular things.

  8. David Eddyshaw said,

    February 13, 2012 @ 1:44 pm

    It's not encouraging that the paper gets the facts wrong straight away with English. Chen says, correctly as far as it goes, that you can't say

    "I go to a seminar" with reference to later today; but of course you might perfectly well say

    "I'm going to a seminar"; indeed I would have thought that is considerably more likely than "I'll go to a seminar" if I'm explaining why I can't attend a meeting later on; in fact tthat would imply (for me) that you were actually deliberately choosing that option, rather than having unfortunately landed yourself in a situation where you had a prior commitment. It would be a snub!

    Of course, you can't say "I go to a seminar" to describe what you're currently doing, either, as it corresponds only to the habitual sense of the present tense of a language like German which doesn't obligatorily distinguish the two. Maybe languages that make compulsory aspect distictions are also the preserve of the feckless …

  9. meesher said,

    February 13, 2012 @ 2:07 pm

    I'm eager to read Mr Chen's analysis of the data on skin pigmentation and GDP.
    For people who want to discount the "soft" social sciences and rely solely on mathematical models, it must be frustrating when they bump up against the vast, largely undocumented history of inter-cultural interactions that would have to be sorted through just to arrive at a conclusion on causality between two features.

  10. David Eddyshaw said,

    February 13, 2012 @ 2:38 pm

    I well remember, though I unfortunately have long since lost the reference, an account of a (one hopes) unusually dense British colonial officer in southern China at the beginning of the last century complaining about the fecklessness of the locals. Doubtless the descendents of these hopeless cases in Hong Kong (who could probably buy out the descendents of the gormless Brit several times over) achieved their current position by dropping the elaborate tense system characteristic of Chinese until the last half century?

  11. Östen Dahl said,

    February 13, 2012 @ 3:46 pm

    I would like to note that I had a fairly extensive correspondence with Chen, who actually invited me to collaborate with him, which I declined. I did try quite hard to dissuade him from the idea, for reasons partly coinciding with what Geoff and Mark have written here. I also pointed out that his diagrams looked more or less equally nice with "rounded front vowels" replacing "weak FTR". The problem is that the countries in previously Protestant, nowadays secularized NW Europe, where both these phenomena are well represented, also contribute heavily to the countries where people are best at saving.

  12. David Eddyshaw said,

    February 13, 2012 @ 3:53 pm

    There are some odd things in Chen's lists of languages as being "strong" or "weak" in their future marking.

    Dutch is "weak", but Afrikaans is "strong." All that English and Bantu influence? Maybe, I suppose.
    Amharic is "weak", but Tigrinya is "strong." Dunno but seems fishy.
    Japanese is "weak", but the structurally very similar Korean is "strong." (Presumably explains why South Korea is such a basket case economically too, compared with the North where the whole verbal system must be different, I guess.) I do know that Korean is well able to use "present" forms with future reference.

    Hard to see the relevance of Irish being "strong." Welsh, by the way used not to have a particular future form but the modern language uses the old present as a futuales was something to do with the loss of our old heavy industry, when all along it was our calamitous decision to invent a future tense.

    Admittedly, picking holes like this, though fun, doesn't mean that Chen's whole edifice is wrong, nor do the easily found individual counterexamples prove that there can't be a correlation in general. It does seem worrying that wherever I'm in a position to check, his facts seem pretty shaky. Doesn't matter how impressive the maths is if the basic data are unreliable. And really, if you are going to maintain that one language in actual use marks the future distinctively significantly more often than another, a glance at a couple of grammars isn't going to give you the answer, especially the sort of fairly basic grammars in his list of references.

  13. Paper Talk: Language and Saving Behavior – REVISITED! « epicnomics said,

    November 23, 2012 @ 2:04 am

    [...] Language Log author created a post pointing out the idea that correlation does not equal causation (along with some very cool visuals to aid his point.) This argument is a more legitimate one, [...]

  14. Saving for a rainy day: Keith Chen on language that forecasts weather — and behavior | Krantenkoppen Tech said,

    February 19, 2013 @ 11:09 am

    [...] skeptical of the work. Their concerns are concisely explained in two well-thought out posts (here and here) by the linguists Mark Liberman and Goeffrey Pullum on the blog they founded, Language [...]

  15. Whorfian economics reconsidered: Why future tense? | Replicated Typo said,

    February 26, 2013 @ 8:55 am

    [...] The hypothesis has been criticised by several linguists, notably on language log (and a great model post by Mark Liberman), where Chen gave a response. The data has been criticised (e.g. English is marked as 'strong [...]

  16. Does Language Influence Our Propensity to Save? | In Sight said,

    April 7, 2013 @ 11:19 am

    [...] Liberman of the University of Pennsylvania highlights the dangers of running statistics on factors that may very well not be completely independent. He [...]

  17. Uncovering spurious correlations between language and culture | Replicated Typo said,

    August 15, 2013 @ 2:17 am

    [...] is the historical relationship between cultures.  Cultural features tend to diffuse in bundles, inflating the apparent links between causally unrelated features.  This means that it's not a good idea to count cultures or languages as independent from [...]

  18. Schuldengrammatik › Sprachlog › SciLogs - Wissenschaftsblogs said,

    November 21, 2013 @ 3:53 am

    [...] [ii] Liberman, Mark (2012) Cultural diffusion and the Whorfian hypothesis. Language Log, 12.2.2012. [Link] [...]

RSS feed for comments on this post · TrackBack URI

Leave a Comment