Wretched analysis, appalling reporting

« previous post | next post »

This is a little lesson in how not to investigate a linguistic question and in how not to use expert opinion about that question. For a change, our target is not BBC News, but instead Wired.com. The piece, by Brian X. Chen, begins promisingly:

The validity of recent e-mails supposedly sent by Steve Jobs to Apple customers is questionable, according to an analysis by Wired.com.

We carefully examined the writing style and grammar of three recent e-mails claimed to have been sent by Jobs with three samples of his confirmed writing.

With help from Wired.com's copy editors and Patrick Farrell, head of the UC Davis linguistics department, we observed that the customer-reported e-mails contained elementary grammatical errors, which are absent from Jobs' real e-mails; the CEO has a much stronger command of the English language than recent e-mails suggest.

It appears, however, that the Wired staff started with a hypothesis, that the e-mail messages were not genuine, based on considerations that had nothing to do with the language of the messages, and then searched for linguistic features that would support this hypothesis, labeling as "grammatical errors" entirely acceptable variants in standard English.

Then, when the article finally gets around to Patrick Farrell, it turns out that what he actually said didn't agree with Wired's opinion:

"The grammar in all the e-mails is competent, native, and standard English," he said.

However, he said the evidence of just three short e-mails was too scant to come to a conclusion.

"I don't see anything obvious that would lead me to believe that the three questionable emails are fake," he said. "I think one would need more evidence. Longer emails or something."

Wretched analysis, appalling reporting.

(The pointer to the Wired piece came from a comment on a Language Log posting on an unrelated topic. One of the bloggers — someone less tolerant than I have been with such things — deleted the comment on the basis of irrelevancy, but sent the pointer to the other bloggers.)

The background, from the Wired piece:

Recently, two Apple customers have said they received e-mails from Jobs: A MacRumors reader claimed Jobs replied to his letter concerning iPhone 3G network issues, and a Gizmodo reader said Jobs wrote him an e-mail hinting at plans to offer an iPhone tethering option. For reference purposes, we also looked at a third e-mail purportedly sent by Jobs regarding the iPod touch.

Though the purported e-mails are terse, they reveal information about what's coming for Apple — which is unusual, since Apple religiously operates in secrecy. Many blogs and news outlets picked up these e-mails and spread around the "facts" as if they came directly from Jobs himself.

The MacRumors e-mail about a fix coming for iPhone 3G to resolve its network issues sparked widespread speculation that the handset was responsible for connectivity problems. As time passed, however, more and more evidence suggests that the 3G networks themselves are more closely tied to network-related problems than the phone.

We compared the three customer-reported e-mails with e-mails verified to be real: Steve Jobs' open letterto iPhone customers, his internal e-mail to Apple staff regarding MobileMe and another internal e-mail he sent to Apple staffers about outperforming Dell.

Note: the number of messages is small (three genuine ones, three in question) and they're all short. Things don't look promising at the outset; unless a questionable message has something flagrantly ungrammatical or idiosyncratic in it, the investigation will be inconclusive.

But it's worse: the features that Wired chose to look at are all acceptable in written standard English. I can only conclude that the Wired staff pored over the questionable messages, looking for something, anything, to find fault with. Here's the list:

(1) Restrictive relativizer which: "We are working on some bugs which affect around 2% of the iPhones shipped, and hope to have a software update soon. Steve"

(2) "Unnecessary commas" in reduced coordination: "We are working on some bugs which affect around 2% of the iPhones shipped, and hope to have a software update soon. Steve"

(3) Reduced coordination: "We are working on some bugs which affect around 2% of the iPhones shipped, and hope to have a software update soon. Steve"

(4) Closing words, like "Sincerely" and "Best regards": "Best, Steve" in one of the three questionable messages (while the three genuine messages have no such closing words).

This is just pathetic. Item (4) isn't about grammar at all, but about a matter of individual writing style. (However, with a much larger sample, it might be taken as a bit of evidence that the "Best, Steve" message was suspect.) Items (2) and (3) concern stylistic choices: the comma in reduced coordination conveys a pause (some copyeditors insist that commas should not be used in such cases — Omit Needless Commas — but they are routinely used that way by excellent writers), and the reduced coordination itself is an option serving brevity (and is of course used by all writers and speakers). The Wired editors would prefer the longer (and perhaps more emphatic):

We are working on some bugs that affected around 2% of the iPhones shipped, and we hope to have a software update soon.

That leaves us with the restrictive relativizer which, which is a famous copyeditors' bugbear. As a result of this history in copyediting, the Wired staff take this feature to be the most consequential one. Chen writes:

Though this analysis is inconclusive, the e-mail containing the misuse of "which" appears to be the most suspicious.

We have posted endlessly on restrictive relativizer which (rather than that) here on Language Log. It is not a mistake in grammar, but a choice available in standard English, a choice made by good writers on occasion (though everyone uses that as well). Once again, the item is a stylistic matter. Unless it turns out that Jobs is someone who never uses which when that would also be possible (such people are rare, in my experience, though a fair number of people believe that they never use which), an instance of which in the questionable messages shows nothing.

As Farrell said, the grammar in all the messages (including the questionable ones) is "competent, native, and standard English". None of them contain "elementary grammatical errors", as claimed in the third paragraph of the Wired piece.

(Note: I am making no claim about the genuineness or otherwise of the three questionable messages.)

[Added 8 September. Bill Poser writes to suggest a reason why the two sets of messages might differ a bit in style (if, indeed, they do): "The three email messages known to be genuine are all public announcements. One is an open letter, the other two are messages to groups of Apple employees. The three unknowns are all messages to a single individual. Of course they will differ stylistically. The public announcements are likely to be more formal and they are likely to have been more carefully vetted."]


Comments are closed.