Language Log

« previous post | next post »

Below is an email message from Steve Mah, posted with his permission. It follows up on my post "It's not easy seeing green", 3/2/2015, about the experiment on Himba color perception shown in the 2011 BBC documentary "Do you see what I see?" (video available here).  I've also appended an earlier email from Jules Davidoff to Paul Kay, telling essentially the same story:  This striking "experiment" was a dramatization, and the description of its "results" was invented by the authors of the documentary, and not proposed or endorsed by the scientists involved.

---

Hello,

I recently read your post on Language Log about a recent Business Insider article that claimed (via a link to BBC documentary footage) that the Himba people of Namibia could not distinguish blue from green.

I was looking into this for my own purposes, and I contacted Serge [Caparos], who was the person shown in the video. He wrote me back as follows:

In 2011, the BBC approached Jules Davidoff about his published colour work (that he did with Debi Roberson between 1998 and 2008). They wanted to send out a team to film something on it. Jules explained to them that they had not done any colour work in Namibia for several years, but that the field site was still active as I was there collecting data on visual attention. So Jules asked me to set up a demonstration for the BBC. The colour work is not actually my work, it's mostly Debi Roberson's, so any question you have might best be sent to her.

When it came to what that demonstration should be, Jules thought that a variation of a visual search paradigm performed with Korean speakers [Debi Roberson et al. "Categorical perception of colour in the left and right visual field is verbally mediated: Evidence from Korean", Cognition 2008] was the simplest procedure for the viewer to grasp. They send me the stimuli for me to show a demonstration. The paper on which the demonstration is based is attached (comparing Korean to English speakers). The work that has been done with the Himba is in reality a bit different to the one that was shown on tv, but this doesn't impact the general message that came across in the BBC documentary (although the prism of tv slightly exaggerates the reality…). The 2005 paper based on the Himba findings is attached. I also put another paper, which reports the findings obtained with Himba children. [(myl) Debi Roberson et al., "The Development of Color Categories in Two Languages: A Longitudinal Study", Journal of Experimental Psychology 2004; Debi Roberson et al., "Color categories: Evidence for the cultural relativity hypothesis", Cognitive Psychology 2005] 

These papers, however, only reported that subjects experienced slower reaction times when distinguishing the oddball colour (or made errors regarding the extent of perceptual distance when it crossed one of their language's colour boundaries). Which is a key difference from the way the experiment was described out in the BBC documentary. I wrote Debi [Roberson], asking her whether she knew of any cases where people failed to distinguish an oddball. She said:

No, you are correct – the differences found have been in the speed of identification of the oddball, and sometimes in a greater number of errors. I'm not aware of any finding (and certainly none with the Himba) where a participant has failed to spot an oddball. Usually, the participant is required simply to report whether the oddball is to the left or to the right of fixation, and when the stimuli are presented in a circular array there are also reaction time differences that depend on where the oddball appears 9 o'clock and 3 o'clock are fastest. RTs get slower for locations closer to 12 o'clock and 6 o'clock. So it is important that each of the tested oddball colours appears equally often in each location (although this is seldom made explicit in descriptions of the methodology).

The work that I did on colour with the Himba pre-dated the work that Serge was involved in, and mostly used paper stimuli – the more recent work that I have done with them has been on other areas of cognition, such as emotional expressions and children's perspective taking, so I wasn't directly involved in the work that was in the video but, as I said, I'm not aware of any reports of complete failure to identify the oddball. Of course, over hundreds of trials even UK participants make errors from time to time though.

---

[Above is email from Steve Mah.]

Earlier, Paul Kay wrote to Jules Davidoff, following up on the same post, to inquire about why the study shown in the documentary hadn't been published. Prof. Davidoff's response:

The power of communication through the internet never ceases to surprise me. I had a similar request only yesterday and many more previously and to all I have had to say there is nothing published. 

The BBC approached us about our published colour work and wanted to send out a team to film something on it. I explained to them that we had not done any colour work for several years but the field site was still active and was sure that Serge who was there carrying out work on attention could set up a demonstration.  When it came to what that demonstration should be, I thought that a variation of your visual search paradigm was the simplest procedure for the viewer to grasp. My assistant Elley Wakui produced the stimuli but Serge will be able to tell you what they were. […] If he does not have the details, I am sure Elley will supply them but I suspect they are the colours you used. We of course did our best to verify colour reproduction.

So to sum the story up as I understand it: The experiment shown in the documentary was a dramatization; the genuine color experiments done with the Himba, some years before, used a different sort of stimuli and a different experimental method; the stimuli shown in the documentary were modeled on those used by Paul Kay and others in experiments on other groups; but in all of the relevant experiments, the dependent measure was reaction time (in finding a matching color or an oddball color), not success or failure.

The BBC's presentation of the mocked-up experiment — purporting to show that the Himba are completely unable to distinguish blue and green shades that seem quite different to us, but can easily distinguish shades of green that seem identical to us — was apparently a journalistic fabrication, created by the documentary's editors after the fact, and was never asserted by the researchers themselves, much less demonstrated experimentally.

This explains why the "experiment" was never published, and why the stimuli shown in the documentary don't make sense.

As a result, the striking and impressive assertions made in the documentary must be completely discounted, and we learn yet again that the BBC deserves shockingly little credibility in reporting on science. I wrote about this a decade ago ("It's always silly season in the (BBC) science section", 8/26/2006), and I don't think that things have gotten any better, though I've given up complaining about it.

 

March 17, 2015 @ 7:14 am · Filed by Mark Liberman under Language and the media

Permalink