Language Log

Despite the panic over AI we're seeing in many sectors of society, including academia, the juggernaut rolls on, seeming set to crush everything in its way:

"EU gives more power to AI translation machines"

The European Commission has launched a pilot project to translate some press releases without any human oversight.

POLITICO (6/15/23)

Three weekends ago, I delivered a keynote here:

New Directions in Chinese Language Education in the 21st Century

The Eighth International Conference on Teaching Chinese as a Second Language

Swarthmore College, June 9-10, 2023

———–

Abbreviations:

    AI — Artificial Intelligence

    DT — Digital Technology

    IT — Information Technology

    DH — Digital Humanities

    AGI — Artificial General Intelligence, where machines supposedly can accomplish any intellectual task that a human can (to me that's a pipe dream)

(given for present and future reference and use)

Title "Aspects of AI and digital technologies in Chinese language teaching"

Abstract

In recent decades, language processing hardware and software have progressed at an astonishing rate, one that is geometric rather than arithmetic.  The opportunities these advances offer and the challenges they pose require our thoughtful attention and careful response, lest the machines get out of control and affect our students in detrimental ways.  DeepL, ChatGPT, and other constantly evolving technologies possess enormous power to manipulate language, power that we can utilize for the enhancement of Chinese language pedagogy.  On the other hand, we must monitor and adapt this potential in such a manner that it fits our purposes and meets the needs of our students. 

A couple of weeks ago, John Hansen tried "an experiment to see if I could successfully combine random and seemingly unconnected topics into one poem", and reported the results on Medium. This experiment was quickly reproduced by Adrian CDTPPW, Block Wife, and Robert G. Longpré.

The buzz about Large Language Models has re-ignited interest in Alan Turing's famous 1950 article "Computing Machinery and Intelligence". Two interesting recent discussions: Jessica Riskin, "A Sort of Buzzing Inside My Head", NYRB 6/25/2023, and Mustafa Suleyman, "The Coming Wave: Technology, Power, and the Twenty-first Century's Greatest Dilemma", Random House 9/5/2023.

Suleyman's book won't be released until 9/5/2023, so it's interesting that several outlets have blurbed one of its ideas ten weeks early: Brad Stone, "AI Leader Proposes a New Kind of Turing Test for Chatbots", Bloomberg 6/20/2023, and Sawdah Bhaimiya, "DeepMind's co-founder suggested testing an AI chatbot's ability to turn \$100,000 into \$1 million to measure human-like intelligence", Business Insider 6/20/2023.  Based just on Business Insider's title, Suleyman's proposal puzzled me, since we don't usually think of machine-trading systems as measuring intelligence — at least not the intelligence of the system rather than its designer. But in fact Suleyman has something different in mind, more along the lines of  an extended "shark tank" competition:

In describing his proposal, Suleyman argues that there’s a misplaced focus in the tech industry on the distant possibility of achieving artificial general intelligence, or AGI: algorithms with cognitive abilities that match or exceed humans’. Instead, he said the more achievable and meaningful short-term goal is what he calls artificial capable intelligence, or ACI: programs that can set goals and achieve complex tasks with minimal human intervention.

To measure whether a machine has achieved ACI, he describes a “modern Turing test” — a new north star for researchers — in which you give an AI \$100,000 and see if it can turn the seed investment into \$1 million. To do so, the bot must research an e-commerce business opportunity, generate blueprints for a product, find a manufacturer on a site like Alibaba and then sell the item (complete with a written listing description) on Amazon or Walmart.com.

Suleyman expects AI will pass this more practical threshold sometime in the next two years. “We don’t just care about what a machine can say; we also care about what it can do,” he writes. And when that happens, he says, “The consequences for the world economy are seismic.”

Article by Melanie Lidman in The Times of Israel (6/17/23):

Groundbreaking AI project translates 5,000-year-old cuneiform at push of a button

‘Google Translate’-like program for Akkadian cuneiform will enable tens of thousands of digitized but unread tablets to be translated to English. Accuracy is debatable.

Opening and key paragraphs:

Cuneiform is the oldest known form of writing, but it is so difficult to read that only a few hundred experts around the world can decode the clay tablets filled with wedge-shaped symbols. Now, a team of archaeologists and computer scientists from Israel has created an AI-powered translation program for ancient Akkadian cuneiform, allowing tens of thousands of already digitized tablets to be translated into English instantaneously.

[This is a guest post by Charles Belov]

I pasted the following Thai, which I got from a YouTube channel, into Google translate. The results were mostly in English, but Google Translate injected some apparent Tamil as well and then just gives up and leaves some of the Thai untranslated.

"ตลอดระยะเวลาการทำงานในวงการบันเทิงมันทำให้เราได้เรียนรู้ว่าจริงๆ เเล้วความสุขอยู่รอบตัวเราไปหมด เเล้วความสุขมันง่ายมาก จริงๆ บางทีความสุขมันก็ไม่ต้องมีเงินเยอะมากมาย ความสุขในชีวิตของผมมันคือการมีอิสรภาพ

ผมรู้สึกว่ามันเเค่ต้อง balance ชีวิตให้มากขึ้น รักตัวเองให้เป็น เงินก็ต้องหา เเต่ก็ต้องให้เวลากับตัวเอง เเคร์ตัวเอง เเคร์คนอื่นน้อยลง"

ฟิล์ม ธนภัทร คนหิวความสำเร็จ กับอิสรภาพของชีวิต

translated to English as:

"During the time of working in the entertainment industry, it made us learn that really, happiness doesn't need much money, so much happiness. in my life it is கெர்பியைப்ப்பு

I feel that you have to find balance in your life, but you have to make time for yourself, take care of yourself, and take care of others less"

Film ตันที่ร ตั้วิที่ สุ้วิต้ามี่ สุ้าวิต้วั่ม

Benj Edwards has a mirthful article in Ars Technica (6/9/23)

Researchers discover that ChatGPT prefers repeating 25 jokes over and over

When tested, "Over 90% of 1,008 generated jokes were the same 25 jokes."

[includes an AI generated image of "a laughing robot"]

On Wednesday, two German researchers, Sophie Jentzsch and Kristian Kersting, released a paper that examines the ability of OpenAI's ChatGPT-3.5 to understand and generate humor. In particular, they discovered that ChatGPT's knowledge of jokes is fairly limited: During a test run, 90 percent of 1,008 generations were the same 25 jokes, leading them to conclude that the responses were likely learned and memorized during the AI model's training rather than being newly generated.

The two researchers, associated with the Institute for Software Technology, German Aerospace Center (DLR), and Technical University Darmstadt, explored the nuances of humor found within ChatGPT's 3.5 version (not the newer GPT-4 version) through a series of experiments focusing on joke generation, explanation, and detection. They conducted these experiments by prompting ChatGPT without having access to the model's inner workings or data set.

…

[Jentzsch and Kersting] listed the top 25 most frequently generated jokes in order of occurrence. Below, we've listed the top 10 with the exact number of occurrences (among the 1,008 generations) in parenthesis:

As I am about to deliver a keynote address to an international conference on Chinese language pedagogy, I receive news of this new LLM that knocks my socks off:

InternLM is a multilingual large language model jointly developed by Shanghai AI Lab and SenseTime (with equal contribution), in collaboration with the Chinese University of Hong Kong, Fudan University, and Shanghai Jiaotong University.

Technical report: [PDF]

Note: Please right click the link above to directly download the PDF file.

Abstract

We present InternLM, a multilingual foundational language model with 104B parameters. InternLM is pre-trained on a large corpora with 1.6T tokens with a multi-phase progressive process, and then fine-tuned to align with human preferences. We also developed a training system called Uniscale-LLM for efficient large language model training. The evaluation on a number of benchmarks shows that InternLM achieves state-of-the-art performance in multiple aspects, including knowledge understanding, reading comprehension, mathematics, and coding. With such well-rounded capabilities, InternLM achieves outstanding performances on comprehensive exams, including MMLU, AGIEval, C-Eval and GAOKAO-Bench, without resorting to external tools. On these benchmarks, InternLM not only significantly outperforms open-source models, but also obtains superior performance compared to ChatGPT. Also, InternLM demonstrates excellent capability of understanding Chinese language and Chinese culture, which makes it a suitable foundation model to support Chinese-oriented language applications. This manuscript gives a detailed study of our results, with benchmarks and examples across a diverse set of knowledge domains and tasks.

From Scott D. Seligman via Facebook:

  ChatGPT is really creeping me out. I asked it for a recipe for Pad Thai in the form of an Emily Dickinson poem. I'm no poetry maven, but the damned thing seems to have the ability to turn a phrase, at least some of the time.

Methodologically, the following communication from Elizabeth J. W. Barber is too important to be left buried in a comment to this post:  "ChatGPT does cuneiform studies" (5/21/23)

As I showed in my 1974 book, Archaeological Decipherment, there is a mathematical algorithm showing how much text one needs to PROVABLY accomplish a decipherment for what sort of script. Since 1974, we haven't added enough new text to our pile of LINEAR A to make it over the hump, if the language it hides is unrelated to anything we already know (or if the hidden language, like Semitic, "cross-classifies" its morphemes between consonants and vowels, since each phonological sign in Linear A represents one C and one V). And if it IS hiding some language we already have a linguistic handle on, we are still scarcely up to the top of the hump. So what language, or language family might one try? We already know that Linear A shows virtually nothing in the way of suffixing or other inflection, so it looks very UN-Indo-European.

Animal communication is not a favorite topic here at Language Log, but according to the following account, one project concerning it seems serious and is being conducted by credible scientists.  Although their claims for its ultimate significance may be inflated, I believe the research they are undertaking is worth considering, especially after hearing the clicks and codas of the sperm whales, which do appear to be communicating data.

Can Understanding Whale Speech Help Us Talk to Aliens?

Biologist David Gruber thinks decoding the language of whales could be just the first step in understanding what other lifeforms are saying—in this world and out of it.

Alexandra Marvar, The Daily Beast (5/13/23)

Here at Language Log, for the last couple months, we've been having long, intense discussions about ChatGPT and other AI chatbots and LLM (Large Language Model) applications.  Now, it seems that the battle over such AI programs has reached the level of ideological warfare.

"America, China and a Crisis of Trust"

Opinion | The New York Times (4/14/23)

Indeed, a story making the rounds in Beijing is that many Chinese have begun using ChatGPT to do their ideology homework for the local Communist Party cell, so they don’t have to waste time on it.

I have some evidence that this might well be true.  Already about half-a-dozen years ago, my M.A. students from the PRC whose parents were CCP members told me that the government required daily interaction with the propaganda installed on their phones — upon pain of being demoted or dismissed.  They had to read a specified amount of Xi-speak and answer questions about the content.  This demanded a serious investment of time (hours).  It was considered to be especially onerous for those CCP members whose day jobs (doctors, bureaucrats, stock brokers, etc., etc.) already demanded a very full work schedule in the office.  So many, if not most of them, hired various human and electronic services to meet the obligations.

Definition

In artificial intelligence (AI), a hallucination or artificial hallucination (also occasionally called delusion) is a confident response by an AI that does not seem to be justified by its training data.

(source)

I had mentioned such AI hallucinating in a previous post once or twice (see "Selected readings"), so it's good to have a concrete example.

Is the account below an instance of ChatGPT "hallucinating"?  Its explanation of gato-por-liebre (cat-for-hare) in Spanish would seem so.

[The following is a guest post by Conal Boyce.]