News Flash: BBC Admits Error
« previous post | next post »
I've pretty much given up criticizing the BBC's reporting on science and technology, since this is Language Log, not BBC-Science-Reporting-Is-Broken Log, and documenting every breathless misunderstanding or credulous reprint of a misleading public-relations handout would take more time than I have available for blogging. So for the past few years, I've examined an occasional bit of BBC-mediated neuro-nonsense, historical hooey or dialectal drivel as if it came from the Daily Mail or the Guardian or any other media outlet from which nothing better should be expected.
But today there's something new: the BBC actually announced, in public, the fact that it had been taken in by a (public-relations?) hoax masquerading as rational inquiry: "Internet Explorer story was bogus", 8/3/2011:
A story which suggested that users of Internet Explorer have a lower IQ than people who chose other browsers appears to have been an elaborate hoax.
A number of media organisations, including the BBC, reported on the research, put out by Canadian firm ApTiquant.
(That should actually be "AptiQuant", if you're keeping score at home.)
This is a strikingly different response than we've seen in the past to bogus BBC reports on cow dialects, "chat-nannies", parrot telepathy, three-headed frogs, breast-enlarging chewing gum, and so on.
I'd like to think that this represents a change in policy; but perhaps the difference is just the size and importance of the interests that their credulity and carelessness threaten in this instance.
One of the reasons for skepticism about the report is the fact that AptiQuant's web site appears to be relatively new, and the pictures of its staff (as well as much of the rest of its site — see below) seem to have been recycled from the web site of a completely unrelated company in France. On the other hand, I found some evidence that AptiQuant did exist at least back in November of 2010. So my tentative evaluation is that the contested study was a PR stunt aimed at promoting the company — like the cow-dialect stunt — and not an elaborate hoax, involving the creation of a completely fake web site, intended to demonstrate the role of sci-tech journalists as transcribers and amplifiers of press releases. But I could well be wrong…
Update — Poking around a bit more on the AptiQuant web site, I find that several (all?) of their pages (for example this one, "Keeping Employees Motivated: The Manager's Role") have been copied from pages on the site of the firm Central Test (in the cited case, this one). Central Test's web site now includes a page denying any connection with AptiQuant. This looks like evidence that the whole AptiQuant company and web site are a hoax — either aimed at promoting this prank web survey, or aimed at some more serious scam.
The plagiarism might also be evidence that AptiQuant's web design team learned their authorial ethics in a recent experience with the modern educational system. But the fact that AptiQuant's "Our Team" page presents exactly the same pictures and layout as Central Testing's "Our Team" page , with just the names changed (and they forgot to change the mouseover text on the CEO!), provides additional evidence that AptiQuant is bogus through and through.
Update #2 — Google News Archive has nothing for AptiQuant, and LexisNexis yields 26 hits, all from 7/29/2011 or later. This is additional evidence that AptiQuant is a hoax, though it doesn't tell us what kind of hoax it is.
A couple of gullible screenshots are here and here.
Update #3 — the perpetrators 'fess up on AptiQuant's web site:
AptiQuant was set up in late July 2011 by comparison shopping website AtCheap.com, in order to launch a fake “study” called “Intelligent Quotient and Browser Usage.” The study claimed that people using Internet Explorer have a below than average IQ score. The study took the IT world by storm. The main purpose behind this hoax was to create awareness about the incompatibilities of IE6, and not to insult or hurt anyone.
They also added a list of "Tell-Tale signs that should have uncovered the hoax in less than 5 minutes":
- The domain was registered on July 14th 2011.
- The test that was mentioned in the report, “Wechsler Adult Intelligence Scale (IV) test” is a copyrighted test and cannot be administered online.
- The phone number listed on the report and the press release is the same listed on the press releases/whois of my other websites. A google search reveals this.
- The address listed on the report does not exist.
- I copy/pasted most of the material from “Central Test” and got lazy to even change the pictures.
- The website is made in WordPress. Come on now!
- I am sure, my haphazardly put together report had more than one grammatical mistakes.
- There is a link to our website AtCheap.com in the footer.
I missed quite a few of these — in particular, I should have checked the domain registration, and I should have noticed that the link in the tag "All rights reserved by AptiQuant" actually went to AtCheap.com. But they don't mention the implausibility of the IQ results.
There's also a page explaining their motivation (persuading people to give up IE6) at greater length. But really, wouldn't it be a good idea to perpetrate similar hoaxes on a regular basis, in order to persuade journalists and editors to be a bit less credulous about scientific, biomedical, and technological press releases?
No, I take it back — most of the hoaxes would never be detected, most of the stories would never be withdrawn, and the wellsprings of public discourse would become even more insalubrious than they are already.
Brett R said,
August 3, 2011 @ 8:56 am
Maybe Microsoft pressured them into making the correction.
[(myl) Maybe. But in general, I think, journalists worry a lot more about their credibility, and are much quicker to withdraw or correct nonsense, when powerful interests are in play — whether the powerful individuals or institutions respond directly or not.]
Barbara Partee said,
August 3, 2011 @ 9:06 am
It was slightly more subtle than that, because it was users of outdated versions of IE that scored lowest, and users of some new kind of IE that I don't know were up in one of the higher groups (above Firefox). It certainly looked plausible enough to me when I looked at it after David Beaver mentioned it around the LLog Plaza water cooler. (I don't know whether it fooled him too or whether he was adding to the hoax.) The basic thesis was that lower IQ users were more likely to stick with what they started with and slower to go for new products or upgrades; it occurred to me that could well correlate with income and free time as well, which they didn't bother to mention but I thought they probably should have …
Jayarava said,
August 3, 2011 @ 9:20 am
Even if it is a hoax, it's not that unlikely. IE is a default option on Windows. Windows is the default OS for most people. Apple users are skewed to higher education and higher income; and Linux is only for geeks (high IQ, low EQ). Changing your browser from the default takes a certain level of intelligence and that skews the average IQ of those who don't change (does anyone actually *choose* IE?). People who don't mess with Windows factory settings are bound to be lower IQ on average (and I stress the *on average*).
patrick said,
August 3, 2011 @ 9:35 am
if you are going to publish something with a trace of a slur against Mircosoft – then this will happen…
graphically reprint a portion of Mickey Mouse's ear and you will be sued before you can say "fair usage"…
mislead the public about those darn pesky scientists with their wunnerful research and you can laugh all the way to the advertising revenue…
D said,
August 3, 2011 @ 9:38 am
If we are going to have a discussion about the actual merits of an admitted hoax, has there been any studies on the correlation or lack thereof between IQ and computer savviness?
Robert Furber said,
August 3, 2011 @ 9:59 am
I notice that the writer is flattering themselves by referring to this as "an elaborate hoax", presumably on the supposition that the BBC would not be fooled by a bleeding obvious hoax, which is exactly what this was.
Chris said,
August 3, 2011 @ 10:05 am
Like Jayarava, I wouldn't be surprised if IE users had a slightly lower IQ than computer users in general, because IE is the default on Windows machines outside of the EU. However, this supposed study showed not just a slight difference, but a massive one. If you look at the chart on p. 4 of their report, they show IQ scores for IE 6 to be in the low 80s. That's about one standard deviation below the mean. That made me skeptical of the results, although I suspected sloppy research methods rather than outright fabrication. (For example, since the IQ tests were supposedly administered via the Internet, one could imagine a question not appearing correctly in IE but appearing fine in other browsers.)
Linda said,
August 3, 2011 @ 10:32 am
They can't even get an unambiguous answer for a University Challenge question. http://www.dailymail.co.uk/news/article-2021841/BBC-defends-University-Challenge-host-Jeremy-Paxman-says-right-answer-wrong.html
Jens Ayton said,
August 3, 2011 @ 12:01 pm
I’m really more concerned about the Cambridge professor interviewed in the BBC’s original article (the Winton Professor of the Public Understanding of Risk, in fact). He was apparently unable to notice the extreme improbability of a ~40 % subset of the population falling more than one standard deviation below the mean, but did feel qualified to that an IQ in the low eighties equates to “borderline deficient, marginally able to cope with the adult world”.
[(myl) Alas, we don't really know what he noticed or didn't notice, but only what BBC News chooses to tell us about his reactions. It would be nice to believe that those two things are roughly the same in essential respects, but such a belief would be naive given known history.]
mefoley said,
August 3, 2011 @ 12:22 pm
I learned soon after I moved to the UK that every time the BBC did a story on something I knew about, they got things wrong. Worrying–how can I trust their reporting on, well, most of what they report about?
I got fed up once and wrote to them, saying that there was no University of Berkeley and if there were, it would probably be in Berkeley, rather than in San Francisco, as their report suggested (adding that a large bay and a couple of other sizeable towns separate the two). I got a yawn-making auto-response. (When they got the name of University of Kentucky wrong and I wrote to them, I got a snotty reply saying pretty much "we can call it whatever we like".)
Still, the BBC is rather like democracy: a horrible system but the best we've found so far.
[(myl) In the areas of science and technology that I know anything about, my impression is that BBC News reporting is by far the worst of any major serious news organization that I regularly read — can you imagine the New York Times or the Economist re-printing a press release about the breast-enlarging effects of some brand of herbal chewing gum, or repeating with a straight face that some parrot's abilities include telepathy?]
Ginger Yellow said,
August 3, 2011 @ 12:53 pm
As MYL suggests, the realissue isn't whether it's a hoax, it's why the BBC (and countless other organisations) would publish the story even if it were a genuine, PR driven bit of "research". I know it's the silly season, but most media organisations (not the Beeb, admittedly) are in the business of charging for advertising, not running it for free as news.
Julie Sedivy said,
August 3, 2011 @ 1:14 pm
AptiQuant seems to have updated their website, including a full confession, an amusing apology for unintentionally "hurting your feelings", and a list of "tell-tale signs" that the study was a hoax.
http://www.aptiquant.com/
Svafa said,
August 3, 2011 @ 3:44 pm
What struck me as odd when I saw the chart was that no numbers were provided. They provide a handy graph to show how users compare in terms of IQ, but do not provide any numbers as to how many users belong to each category. I would think the sample size of each category was fairly important to evaluation, particularly considering the outliers comprise a small percentage of users.
IE6 has dropped below 5% (~3.5%) of the global userbase according to StatCounter. My own observation from the logs of the websites I work on is that IE6 users comprise less than 1% of our total traffic. I've often seen similar claims from other developers and administrators on the various fora I peruse and participate in. The stated total sample size for the study was 101,326 users, so I believe we could reasonably conclude that IE6 users would comprise at most around 3,500 users, but possibly as low as 1,000 or fewer. I'd still expect a result closer to average in a group that size, but a few individuals can heavily weigh the scales to one end or the other with so few.
In comparison, Opera and Camino aren't even listed among the top 12 on StatCounter's usage graph for browser versions, both being relegated to "other" which comprises approximately 6% altogether. If viewing their top 5 browsers (irrespective of version) we find that Opera claims 1.66% of the global userbase, less than half their statistic for IE6. Camino doesn't make the cut.
Anyway, that's a decent amount of background to simply point out the ridiculousity apparent in the survey when I saw it. Especially considering it's a hoax and has been revealed as such at this point. However, I wouldn't expect someone outside the IT field to know this sort of information off the top of their head, so I provided some research rather than simply stating it as fact. Hopefully the information is useful in the larger discussion.
@Barbara
The "new kind of IE" you mention isn't IE or even a separate browser really. It's a plug-in (think Flash or Silverlight) that runs Chrome inside IE. It's primary use, at least to my knowledge, is to artificially upgrade a browser when the user lacks permissions to do so. For an example of such a situation, several government offices, insurance companies, and even some institutes of higher education restrict users to IE6 without the ability to update their browser. Using the Chrome Frame plug-in circumvents this and allows them to browse websites as though they were using Chrome.
Dave Orr said,
August 3, 2011 @ 5:09 pm
A happy side effect: this has spawned a lovely crash blossom:
Internet Explorer Dumb Users Report a Hoax
The dumb users reported a hoax? How did they know it was a hoax if they were so dumb?
http://www.stuff.co.nz/technology/digital-living/5386884/Internet-Explorer-dumb-users-report-a-hoax
Ken Brown said,
August 3, 2011 @ 6:00 pm
I feel strangely compelled to point out that the BBC makes some excellent TV and radio programmes about science. Its their news reporting that is embarrassing crap.
Ginger Yellow said,
August 4, 2011 @ 4:14 am
Ken – it also makes some terrible TV programmes about science, namely 90% of Horizon and quite a few Panoramas.
John F said,
August 4, 2011 @ 4:33 am
I found Men Who Stare At Goats particularly funny because the BBC had produced an Horizon programme in the 90s that took remote viewing a bit too seriously. Homeopathy, too.
I get science news from Ars Technica, so I hardly ever bother with the BBC's science reporting.
RP said,
August 4, 2011 @ 6:19 am
When I notice errors on the BBC news site, I tend to report them at http://news.bbc.co.uk/newswatch/ukfs/hi/newsid_3950000/newsid_3955200/3955223.stm . I've found that usually they get corrected and I sometimes get a personal email telling me so (even though they say not to expect a reply). However, my corrections have never been to the science pages. Also, they have always been corrections to details within a story, rather than questioning the basis for the whole story (as is the case with many of MYL's examples). Still, I would advise people who notice a mistake in a story to report it (if you don't already).
David S said,
August 4, 2011 @ 9:48 am
Yikes, bracketing the Guardian in with the Daily Mail! What's wrong with them, Mark? I agree with your point about the high standards of the NYT and the Economist, but in my experience the Guardian's reportage is not far behind, and is streets ahead of most of the English language print media.
[(myl) OK, that was provocative at best and actually rather unfair. The Guardian does publish Ben Goldacre. But then there's this kind of thing as well. On the other, I have to admit that they are low on telepathic parrots and breast-enlarging herbal-chewing-gum stories.
In fact, the science coverage in the NYT is often questionable, e.g. here. And at least one of their main science writers is proud, more of less, of his role as a purveyor of misinformation.]
thom said,
August 5, 2011 @ 4:37 am
@Jens Ayton
I think Spiegelhalter's comments are perfectly reasonable – it is quite possible for a large subset of the reported data to fall 40% below the mean if they data collection and analysis are shoddy. I recently saw a published paper in which over 60% of participants were above median on one measure. He clearly spotted that the results were implausible and therefore the research questionable … There are so many flaws in the 'research' it would be unreasonable to expect him to have listed them all (though he may well have spotted more than is reported).