Language Log

« previous post | next post »

A couple of days ago, when I posted about Nicholas Kristof's take on the electrophysiology of politics, I limited my discussion to a 2008 Science article about the relationship between physiological reactions to "threatening images" (like a spider on someone's face) and political attitudes towards "protective policies" (like immigration). Thanks to a couple of enterprising readers, I found a readable .pdf of a 2009 manuscript from the same (University of Nebraska) laboratory, which Kristof also discusses, on the relationship between physiological reactions to "disgusting images" and attitudes towards "sex items" like gay marriage. And as promised, I have a bit more to say after reading it.

Taken together, the two papers are more convincing than either one alone, since the combined results call into question the idea that the measured physiological differences might simply be due to the greater uneasiness of (more conservative) townies being hooked up to electrodes in a university laboratory, compared to (more liberal) university faculty, students, and staff. But the second paper confirms some of the concerns that I had about the size of the effects in the first paper. And it also shows that I was off base when I wrote that "This is not a case of egregious journalistic misunderstanding or over-interpretation". In particular, Kristof doesn't just exaggerate, he directly contradicts the 2009 paper's findings when he writes that

Liberals released only slightly more moisture in reaction to disgusting images than to photos of fruit. But conservatives’ glands went into overdrive.

The 2009 paper is Smith et al., "The Ick Factor: Physiological Sensitivity to Disgust as a Predictor of Political Attitudes". The subjects in the study appear to be the same 50 Lincoln-area people used in the 2008 Science paper, and in fact, the data in the two papers seem to have come from the different aspects of the same two sessions, one involving answers to several political and psychological questionnaires, and the other involving measurements of physiological responses to seeing a series of still images. (Perhaps this is wrong, and it's just that the time period, the procedures, and the number of subjects are so similar as to give this impression.)

Political attitudes were measured by ascertaining reactions to16 brief issue-prompt items presented in the well-known Wilson-Patterson format (Wilson and Patterson 1968) in which respondents indicate agreement or disagreement (they can also equivocate). The specific prompts employed … and cover a range of topics including taxes, defense, and social issues.

The three possible answers were coded as 0, 0.5, and 1, in such as way as to make the more (conventionally) "conservative" position equal to 1.

To measure self-report, we employed the standard disgust sensitivity survey battery, DS-R (Disgust Sensitivity-Revised). […] [A] full discussion of this widely-used battery can be found at the Disgust Scale Homepage maintained by Jonathan Haidt.

To measure physiological response to disgust, we relied upon one of the most common indicators of physiological activity: skin conductance. […]

In order to stimulate possible disgust reactions, participants were shown five individual disgusting still images scattered throughout the sequential showing of a large number of individual images. Each image appeared on a computer screen for 15 seconds and was separated from the succeeding image by an inter-stimulus interval (ISI) of 10 seconds. Subsequent factor analysis indicated that responses to three of the five disgusting stimuli loaded heavily on the same factor, so our central measure of physiological disgust sensitivity is based on readings obtained during viewing of these three identified disgusting images: an emaciated body, human excrement, and a person eating a mouthful of worms.

One thing that struck me as odd about this is two of the three "threatening" images used in the 2008 Science paper also strike me as having a substantial "ick factor": the authors describe them as "the face of a frightened person, a dazed individual with a bloody face, and an open wound with maggots in it". And many people might find the "disgusting" images somewhat "threatening" as well, it seems to me. (This will become relevant to the discussion later on.)

In any case, the results show clearly that Kristof's statement ("Liberals released only slightly more moisture in reaction to disgusting images than to photos of fruit. But conservatives’ glands went into overdrive.") is not just an exaggeration of the size of the effect, as I suggested in my earlier post, but a qualitatively false assertion. Here's the table showing correlations of subjects' phsyiological responses to the "disgust" images with their political attitudes:

The overall correlation with conservative attitudes in general (represented by the "index of all 16 items") is a small (and statistically insignificant) 0.18 — and falls to 0.14 if gender and age are controlled for.  6 of the 16 items have negative correlations, meaning that the glands of people who responded as "liberals" on those items tended to respond more strongly than those who responded as "conservatives" (though the difference was not judged to be statistically significant).  8 of the remaining 10 items had positive but statistically insignificant correlations. The only questionnaire items with significant positive correlations to the disgust-picture skin-conductance responses were "Gay Marriage" and "Premarital Sex".

Now, this calls into question my suggestion that perhaps "conservatives" responded more strongly (to the "threatening" pictures) because they were in an environment that made (at least some of) them (at least somewhat) more ill at ease than the "liberals" were. In this data, people with conservative positions on Small Government, Illegal Immigrants, Military Spending, and the Death Penalty, had (on average) weaker SCL reactions than people with liberal positions on those issues did. Of course, the data is clearly very noisy, and so maybe this is all just noise.

But still, the pattern undermines the "social distance" hypothesis, and it flatly contradicts Kristof's asssertion about those conservative glands going into overdrive.

At this point, I'd like to reiterate the point that we're dealing with between-group differences that are small relative to the amount of within-group variation, even in the case of the strongest observed connections between physiology and politics. In my earlier post, I made this point by calculating the difference in means in terms of pooled standard deviation, and showed what sort of distributional overlap this implies. This time, I'll do it in terms of correlations.

The strongest correlation observed in this study is r=0.42, between SCL responses to "disgusting" pictures and attitudes towards Gay Marriage. What does this mean? Graphically, if we had two continuous variables, and plotted one of them against the other, the scatter-plot would look something like this:

This is fake data, alas, since the actual underlying results are not published. I created it with this little script in the R statistics language. (Sorry about the lack of indentation — WordPress's brain-dead interface eats all line-initial spaces…)

C=0
NoiseGain=50
while(C != 0.42){
X1 = 1:48 + NoiseGain*runif(48)
X2 = 1:48 + NoiseGain*runif(48)
C = round(cor(X1,X2),digits=2)
}
plot(X1,X2, type="p", pch="x", col="red",
main="48 pairs of points, illustrating r=0.42\n(Fake data)",
xlab="Skin-conductance effect of 3 disgusting pictures",
ylab="Opposition to gay marriage",
yaxt="n", xaxt="n",
mgp=c(1,1,1))

To be more accurate, I should quantize the political-attitude results into the three bins of their survey responses. Doing that, we get an illustration like this one:

The R code:

C=0
while(C != 0.42){
X1 = 1:48 + NoiseGain*runif(48,min=-0.5,max=0.5)
X2 = 1:48 + NoiseGain*runif(48,min=-0.5,max=0.5)
X2[X2<=16] = -1
X2[X2>16 & X2<=32] = 0
X2[X2>32] = 1
C = round(cor(X1,X2), digits=2)
}
plot(X2,X1, type="p", pch="x", col="red",
main="48 pairs of points, illustrating r=0.42\n(Fake data)",
ylab="Skin-conductance effect of 3 disgusting pictures",
xlab="Opposed to gay marriage? (NO, INDIFFERENT, YES)",
yaxt="n", xaxt="n",
mgp=c(1,1,1)

Again, this is not the real data — I'd love to have the real data to show you, but anyhow, given a correlation of r=0.42, it must look something like this.  And again, you can see that the range of skin-conductance effects overlaps very substantially across the categories of political attitudes.  It's probably a real effect, unlikely to have arisen through sampling error — though I'd feel better about this if I was sure that they'd done a correction for multiple comparisons, and if the set of "disgusting" pictures involved hadn't been adjusted post hoc:

In order to stimulate possible disgust reactions, participants were shown five individual disgusting still images scattered throughout the sequential showing of a large number of individual images.[…]. Subsequent factor analysis indicated that responses to three of the five disgusting stimuli loaded heavily on the same factor, so our central measure of physiological disgust sensitivity is based on readings obtained during viewing of these three identified disgusting images.

I should emphasize that a correlation of 0.42 is nothing to be ashamed of. As I noted in a post a few months ago, a metanalysis of research results in Social Psychology shows that the mode of published correlations is below r=0.1, and 0.42 is well out on the upper tail:

But the point is that we're dealing with small-to-moderate shifts in heavily-overlapping distributions, not some sort of partition of the population into purity-seeking conservatives and ick-tolerant liberals (if that's even the right description for what's happening here).

And now the question of their variables really mean comes to the fore. There are a few things that make this less clear than one would like it to be.

The most important puzzle is the strange divergence between their physiological and psychological measures of disgust sensitivity. Recall that they also "employed the standard disgust sensitivity survey battery, DS-R (Disgust Sensitivity-Revised)" to measure their subjects' self-reported disgust sensitivity. This measure also correlated with attitudes toward Gay Marriage (and to a lesser extent with other sex-related political attitudes):

So what comes next is unexpected :

The roughly parallel results for physiological and for self-reported disgust sensitivity naturally lead to suspicions that the two measures correlate strongly with each other. Surprisingly, however, despite the fact that both measures are related to political attitudes involving sexuality, self-reported disgust sensitivity is unrelated to physiological disgust sensitivity (as measured by skin conductance changes caused by the viewing of disgusting stimuli). The simple bivariate correlation of the two is substantively small, inverse (r = -.15), and statistically insignificant (p = .32).

Note also that self-reported disgust sensitivity also correlated with liberal attitudes on Welfare Spending, Small Goverment, Foreign Aid, and Gun Control. These effects were reduced but not eliminated by controlling for sex and age, with sex playing the most important role — women tended to have higher self-reported disgust sensitivity than men:

One possible reason for the absence of a bivariate relationship is the potentially confounding effect of gender. It may be that differences (physiological and otherwise) between males and females need to be controlled in order for a relationship between physiological and self-reported disgust sensitivity to appear.

Interestingly, while the data suggest important differences between males and females, these differences are not physiological and they do not seem to be the reason for the absence of a correlation. As indicated in Figure 1, where the range of both physiological and self-reported disgust sensitivity has been standardized to run from 0 to 1 in order to facilitate comparisons, mean gender differences occur for self-reported disgust sensitivity (p < .01) but not for physiological disgust sensitivity (p = .82). Perhaps females claim to be more disgust sensitive because, due to societal norms, they often feel pressure to be disgust sensitive whereas males often feel pressure to be disgust insensitive. Though this interpretation is only speculation, we can state with some certainty that, when the focus of attention is on physiology, the difference between males and females is substantively minute and statistically insignificant. When the focus is on self-report, however, males claim to be substantially and significantly less sensitive to disgust than females. Previous scholarship (and, undoubtedly, folk wisdom) holding that “women are more disgust sensitive than men,” (Inbar, Pizarro, and Bloom 2009) is more accurately summarized as women report being more disgust sensitive than men.

Here's their Figure 1:

Note that (as discussed here) this is typical of sex differences on other emotional dimensions such as "empathy":

In general, sex differences in empathy were a function of the methods used to assess empathy. There was a large sex difference favoring women when the measure of empathy was self-report scales; moderate differences (favoring females) were found for reflexive crying and self-report measures in laboratory situations; and no sex differences were evident when the measure of empathy was either physiological or unobtrusive observations of nonverbal reactions to another's emotional state.

The sex/gender – self-report/physiology stuff is interesting, but as the authors explain, it doesn't account for the unexpected disconnection between their measures:

These gender differences in self-report are substantial, however they do not account for the lack of overall correlation between reported disgust sensitivity and physiological disgust sensitivity. A partial correlation of reported and physiological disgust sensitivity controlling for the effects of gender still does not produce a statistically significant relationship (r = -.18; p = .23) and exactly the same numbers result when both gender and age (another variable that could rightly be thought to affect physiological readings) are included as control variables (r = -.18; p = .23). In sum, individuals who believe themselves to be particularly disgust sensitive or particularly disgust insensitive are unlikely to be reporting sentiments that in point of fact relate to their physiological responses to disgust. People are not particularly adept at reporting their emotional states and the gender differences in self-report but not in physiology encourage the conclusion that societal pressures could be more crucial in shaping self-reports than physiological responses.

So in the end, I'm left wondering what space of "dispositional temperaments" is really at issue here, and what its relationship is to the measurements that they're using.

If you remove all the interpretations and just look at the data in this study, we've got something like a 48 X 24 matrix (48 subjects by 16 political questions plus physiological responses to 5 images plus the DS-R measure plus sex plus age). There are more than 200 pairwise correlations, plus a much larger number of potential multivariate analyses. A few of these turn out to be statistically significant. (After Bonferroni correction? I'm not sure.) Specifically, if we look at some of the inter-relationships among various of the political questions, subject sex and age, reactions to three of the pictures, and the DS-R measure, we get a pattern that partly makes sense in terms of Haidt's theory of purity-conscious conservatives, and partly (maybe mostly?) doesn't.

To the extent that I understand all of this, the results seem at least as puzzling as they are intriguing. And given how murky the political, psychological, and physiological spaces are, and how weak the interconnections tend to be, it worries me that most of the people who are thinking in public about these things tend to talk about them in the essentialist terms implied by the use of generic plurals ("… conservatives … liberals …")  This includes not only people like Kristof, who probably don't know any better, but also people like Jonathan Haidt and the authors of this study, who clearly do.

February 18, 2010 @ 10:05 am · Filed by Mark Liberman under Language and politics, Language of science

Permalink