Language Log

Just a little over a year ago, I made the following post:

"The future of Chinese language learning is now"  (4/5/14)

The second half of that post consisted of an account of a lecture that David Moser (of Beijing Capital Normal University and Academic Director of Chinese Studies at CET Beijing) had delivered a few days earlier (on 4/1/14) at Penn:  "Is Character Writing Still a Basic Skill?  The New Digital Chinese Tools and their Implications for Chinese Learning".

Phishers trawling for email account names are generally smart enough to pull all sorts of programming tricks, forging headers and obtaining lists of spammable addresses and setting up arrangements to capture login names and passwords obediently typed in by the gullible; but then they give themselves away with errors of grammar and punctuation that are just too gross to be perpetrated by the authorized guys at the communications and technology services unit.

I received a phishing spam today that had no To-line at all (none of that "undisclosed recipients" stuff, and no mention of my email address in it anywhere). It looked sort of convincing in its announcement that webmail account holders would have to take certain steps to ensure the preservation of their address books after being "upgraded to a new enhanced Outlook interface". (My own university has, tragically, been induced to do an upgrade of this kind to its employee email services.) But the linguistic errors in the message begin with the 13th character in the From line (that second comma is wrong). I reproduce below the raw text of what I received, stripping out only the locally generated receipt and spam-checking headers (and by the way, this message—spam though it is—succeeded in getting a spam score of 0).

Guest post by Karen Stollznow

In recent weeks we've been following the tragedy and mystery of the Malaysia Airlines flight 370 that vanished on March 8 with 239 people on board. Less than an hour after taking off from Kuala Lumpur en route to Beijing all communication was cut off. The plane diverted unexpectedly across the Indian Ocean and disappeared from civilian air traffic control screens. There has been much controversy surrounding the transcript of the last incoming transmission between the air traffic controller and the cockpit of the ill-fated flight.

We tend to have a morbid fascination with people's last words. We assign profound meaning and philosophical insights to the final words uttered by those who face their fate ahead of us. There are numerous books and websites that chronicle the linguistic legacies of famous people such as Douglas Fairbank's ironic, "I've never felt better," to Woodrow Wilson's courageous, "I am ready," and the betrayal expressed in Julius Caesar's "Et tu, Brute?" Planecrashinfo.com maintains a database of last words from cockpit recordings, transcripts, and air traffic control tapes. These are disturbing announcements of impeding doom, including: "Actually, these conditions don't look very good at all, do they?" through to an assortment of cuss words, and moving farewells like, "Amy, I love you."

Gary Marcus and Ernest Davis say in a New York Times piece on why we shouldn't buy all the hype about the Big Data revolution in science:

Big data is at its best when analyzing things that are extremely common, but often falls short when analyzing things that are less common. For instance, programs that use big data to deal with text, such as search engines and translation programs, often rely heavily on something called trigrams: sequences of three words in a row (like "in a row"). Reliable statistical information can be compiled about common trigrams, precisely because they appear frequently. But no existing body of data will ever be large enough to include all the trigrams that people might use, because of the continuing inventiveness of language.

To select an example more or less at random, a book review that the actor Rob Lowe recently wrote for this newspaper contained nine trigrams such as "dumbed-down escapist fare" that had never before appeared anywhere in all the petabytes of text indexed by Google. To witness the limitations that big data can have with novelty, Google-translate "dumbed-down escapist fare" into German and then back into English: out comes the incoherent "scaled-flight fare." That is a long way from what Mr. Lowe intended — and from big data's aspirations for translation.

When I began learning Mandarin nearly half a century ago, I knew exactly how I wanted to acquire proficiency in the language.  Nobody had to tell me how to do this; I knew it instinctively.  The main features of my desired regimen would be to:

1. pay little or no attention to memorizing characters (I would have been content with actively mastering 25 or so very high frequency characters and passively recognizing at most a hundred or so high frequency characters during the first year)

2. focus on pronunciation, vocabulary, grammar, particles, morphology, syntax, idioms, patterns, constructions, sentence structure, rhythm, prosody, and so forth — real language, not the script

3. read massive amounts of texts in Romanization and, if possible later on (after about half a year when I had the basics of the language nailed down), in character texts that would be phonetically annotated

In late 2012, while visiting my son Tom in Dallas, I noticed that he was doing something very odd with his cell phone.  Most people enter text into their cell phone by pressing their thumbs (or their fingertip) on the letters of a small keyboard, whether virtual or actual.  But Tom was doing something altogether different:  he was sliding his finger over the glass surface of his phone and somehow, by so doing, he was able to enter text.  I was dumbfounded!  What amazed me most of all was how casual he was about it.  He'd be talking to me about something, then glance down at his cell phone, move his fingertip around on the glass, and — presto digito! — he'd have typed a message to someone and sent it off.

I recently heard of another friend-of-a-friend case in which people were taken in by one of the false email help-I'm-stranded scams, and actually sent money overseas in what they thought was a rescue for a relative who had been mugged in Spain. People really do respond to these scam emails, and they lose money, bigtime. Today I received the first Nigerian spam I have seen in which I am (purportedly) threatened by the FBI and Patriot Act government if I don't get in touch and hand over personal details that will permit the FBI to release my $3,500,000.

I wish there was more that people with basic common sense could do to spread the word about scamming detection to those who are somewhat lacking in it. The best I have been able to do is to write occasional Language Log posts pointing out the almost unbelievable degree of grammatical and orthographic incompetence in most scam emails. Sure, everyone makes the odd spelling mistake (childrens' for children's and the like), but it is simply astonishing that literate people do not notice the implausibility of customs officials or bank officers or police employees being as inarticulate as the typical scam email.

The one I just received is almost beyond belief (though see my afterthought at the end of this post). The worst thing I can think of to do to the senders is to publish the message here on Language Log, to warn the unwary, and perhaps permit those who are interested to track the culprit down. I reproduce the full content of the message source below, with nothing expurgated except for the x-ing out of my email address and local server names. I mark in red font the major errors in grammar and punctuation, plus a few nonlinguistic suspicious features.

What xkcd is getting at with the latest comic is about syntax and semantics. I'll show you the syntax below, but as far as meaning is concerned, the point is that cell phone numbers have almost no semantics. The area code part (the first three digits) used to function as a locational marker when phones were in fixed locations in houses, but since Americans not only tend to move every three years or so but they now take phone numbers with them, and cell phone universality only really began to pick up in America five to ten years ago, it really does tend to reflect a former abode. My cool son Calvin, for example, has a number which implies that he lives in Oakland, California; he doesn't, he does his video game programming in the Pacific North West.

And the rest of the number, the other seven digits? Space enough there for some real personal information, but it is not used. It functions merely as arbitrary material to distinguish one cell phone's location point in the information universe from all the others.

There's a passage in James Gleick's "Auto Crrect Ths!", NYT 8/4/2012, that's properly spelled but in need of some content correction:

If you type “kofee” into a search box, Google would like to save a few milliseconds by guessing whether you’ve misspelled the caffeinated beverage or the former United Nations secretary-general. It uses a probabilistic algorithm with roots in work done at AT&T Bell Laboratories in the early 1990s. The probabilities are based on a “noisy channel” model, a fundamental concept of information theory. The model envisions a message source — an idealized user with clear intentions — passing through a noisy channel that introduces typos by omitting letters, reversing letters or inserting letters.

“We’re trying to find the most likely intended word, given the word that we see,” Mr. [Mark] Paskin says. “Coffee” is a fairly common word, so with the vast corpus of text the algorithm can assign it a far higher probability than “Kofi.” On the other hand, the data show that spelling “coffee” with a K is a relatively low-probability error. The algorithm combines these probabilities. It also learns from experience and gathers further clues from the context.

The same probabilistic model is powering advances in translation and speech recognition, comparable problems in artificial intelligence. In a way, to achieve anything like perfection in one of these areas would mean solving them all; it would require a complete model of human language. But perfection will surely be impossible. We’re individuals. We’re fickle; we make up words and acronyms on the fly, and sometimes we scarcely even know what we’re trying to say.

Yesterday I went to the Beijing Public Security Bureau (Gōng'ān jú 公安局) to renew my visa.  While waiting in the main hall for my number to be called, I had ample time to walk around and familiarize myself with the operations there.  One thing in particular piqued my curiosity.  Namely, I saw four gray, metal cabinets full of hundreds of passports (three for Chinese, one for foreigners) waiting to be picked up.

I watched a clerk filing passports into the slots on the mechanized, revolving shelves inside the cabinets.  Wondering how the passports were arranged so that they could be readily retrieved when called for, I asked the supervisor how the passports were ordered on the shelves.  Her reply left me both startled and pleased.

Under the above rubric, my friend Apollo Wu sent around a note (copied below) about the economic impact of the use of Chinese characters in the operation of his business.  Since Apollo was for many years (from 1973 to 1998) a top translator in the Chinese Translation Service at United Nations headquarters in New York, he knows whereof he speaks.  Among other interesting tidbits that I heard from Apollo over the decades was that, of the official languages of the United Nations (Arabic, Mandarin Chinese, English, French, Russian, and Castilian Spanish) Chinese was by far the least efficient and most expensive to process.

We neglected to mention this while the relevant cartoon was the current one at xkcd, but a couple of days ago there was a nice analysis of why through 20 years of effort, we've successfully trained everyone to use passwords that are hard for humans to remember but easy for computers to guess. Check it out. The observation seems correct: if you try it out on one of the web interfaces that assess the strength of your password as you choose it, you'll find that a word with a few letters replaced by miscellaneous digits and so on, like Ne8r@$k@, gets high marks but grizzle snip grunt mackerel doesn't (and probably won't be accepted beyond the first 8 to 12 characters). Yet if you mutter "grizzle snip grunt mackerel" under your breath once, you'll find you remember it all day, even without using it. And length is your main security. The example the cartoon gives contrasts a 3-day brute-force cracking time (for about 28 bits of entropy) with a 550-year time (for about 44).

I got a royalty check from Chicago today, and I stared in astonishment at the home address on the payment advice. It was roughly correct in the first four lines, but the last line, after "EDINBURGH EH3 6RY", where the country name "United Kingdom" should have come, said "TAIWAN, PROVINCE OF CHINA".