Language Log

Whisper is a widely-used speech-to-text system from OpenAI — and it turns out that generative AI's hallucination problem afflicts Whisper to a surprisingly serious extent, as documented by Allison Koenecke, Anna Seo Gyeong Choi, Katelyn X. Mei, Hilke Schellmann, and Mona Sloane,"Careless Whisper: Speech-to-Text Hallucination Harms", In The 2024 ACM Conference on Fairness, Accountability, and Transparency,  2024:

Abstract: Speech-to-text services aim to transcribe input audio as accurately as possible. They increasingly play a role in everyday life, for example in personal voice assistants or in customer-company interactions. We evaluate Open AI’s Whisper, a state-of-the-art automated speech recognition service outperforming industry competitors, as of 2023. While many of Whisper’s transcriptions were highly accurate, we find that roughly 1% of audio transcriptions contained entire hallucinated phrases or sentences which did not exist in any form in the underlying audio. We thematically analyze the Whisper-hallucinated content, finding that 38% of hallucinations include explicit harms such as perpetuating violence, making up inaccurate associations, or implying false authority. We then study why hallucinations occur by observing the disparities in hallucination rates between speakers with aphasia (who have a lowered ability to express themselves using speech and voice) and a control group. We find that hallucinations disproportionately occur for individuals who speak with longer shares of non-vocal durations—a common symptom of aphasia. We call on industry practitioners to ameliorate these language-model-based hallucinations in Whisper, and to raise awareness of potential biases amplified by hallucinations in downstream applications of speech-to-text models.

This paper's content is interesting — Mirzadeh, Iman, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. "GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models." arXiv preprint arXiv:2410.05229 (2024). In short, the authors found that small changes in Grade-School Mathematics benchmark questions, like substituting different numerical values or adding irrelevant clauses, caused all the tested LLMs to do worse. You should read the whole thing for the details, to which I'll return another time.

Things are happening very fast with this "experimental" internet search tool.  I realize that it is a new technology, so naturally there are bugs and kinks that have to be worked out, and I don't want to be too harsh with it.  Moreover, at a certain level, it is already serving a yeomanly purpose.

For instance, I asked Google, "should ich be capitalized in the middle of a sentence".  AI Overview (henceforth AIO) promptly stepped in and provided the following straightforward response:

No, ich should not be capitalized in the middle of a sentence in German. In German, the first-person singular pronoun ich (I) is only capitalized when it is the first word in a sentence.

In English, the singular “I” is always capitalized when used as a first-person personal pronoun. This includes all contractions of “I”, such as “I'm” and “I'll”.

Other things to capitalize in the middle of a sentence include…. [details omitted here]

[N.B.:  If you don't have time to read through this long and complicated post, cut to the "Closing note" at the bottom.]

Lately when I do Google searches, especially on obscure and challenging subjects, AI Overview leaps into the fray and takes precedence at the very top, displacing Wikipedia down below, and even Google's own responses, which have been increasingly frequent in recent months, are pushed over to the top right.

AI Overview, on first glance, seems convenient and useful, but — when I start to dig deeper, I find that there are problems.  As an example, I will give the case of the name of the Snake River, and maybe mention a few other instances of AI Overview falling short, but still being swiftly, though superficially, helpful.

Google has created an experimental — and free — system called NotebookLM. Here's its current welcome page:

Elle Cordova offers an update from ChatGPT on the number of Rs in "strawberry":

Daysia Tolentino, "Trump shares fake photo of Harris with Diddy in now-deleted Truth Social post", NBC News 9/20/2024:

Amid the recent news of Sean “Diddy” Combs’ arrest, former President Donald Trump reposted a doctored image falsely showing Vice President Kamala Harris with Combs with text questioning if she was involved in his alleged “freak offs.”

The image, which Trump reposted to his Truth Social profile, is an edited version of a 2001 photo of Harris with former talk show host Montel Williams, whom she briefly dated, and his daughter Ashley. The edit replaced Montel Williams’ face with a photo of Combs.

This is not the first time the Republican presidential nominee has posted a fake image in an effort to bolster his campaign. Trump has posted several AI-generated images, including some falsely depicting Taylor Swift and her fans endorsing him, and one of Harris speaking to a crowd of communists in Chicago during the Democratic National Convention.

Apparently not. Given this recent tweet, in which Google AI Overview explains that "October 21 is not a Libra, as the Libra zodiac sign is from September 23 to October 22", I thought I'd try for myself. The result had a different format but the same problem:

So says DeepL CEO Jarek Kutylowski.

"DeepL translation targets Taiwan as next key Asian market:  CEO says AI-based model is aiming to refine nuances, politeness", Steven Borowiec, Nikkei staff writer (September 16, 2024)

DeepL Write is one thing, DeepL Translator is another.  We've examined both on Language Log and are aware that the former is already deeply entrenched as a tool for composition assistance, but are less familiar with the special features of the latter.

The article by Borowiec, based on his interview with CEO Jarek Kutylowski, begins with some not very enlightening remarks about the difference between simplified characters on the mainland and traditional characters on Taiwan, attesting to the truism that CEOs and CFOs often don't know as much about the nitty-gritty technicalities of the products they sell as do the scientists and specialists they hire to make them.

An eminent Chinese historian just sent these two sentences to me:

Yǒurén shuō AI zhǐ néng jìsuàn, ér rénlèi néng suànjì. Yīncǐ AI yīdìng bùshì rénlèi duìshǒ

有人說AI只能計算，而人類能算計。因此AI一定不是人類對手。

"Some people say that AI can only calculate, while humans can compute.  Therefore, AI must not be a match for humans".

Google Translate, Baidu Fanyi, and Bing Translate all render both jìsuàn 計算 and suànjì 算計 as "calculate".  Only DeepL differentiates the two by translating the latter as "do math".

"AI worse than humans in every way at summarising information, government trial finds:
A test of AI for Australia's corporate regulator found that the technology might actually make more work for people, not less."  Cam Wilson, Crikey (Sep 03, 2024)

The transcriptions on YouTube are generally pretty good these days, but sometimes the results are weird.

A notable recent example is the transcription of Donald Trump's 8/31/2024 Fox interview with Mark Levin, where the system renders "Putin" first as "pollutant" and then as "pooch and".

It's been clear for a while that "large language models" can be prompted to fulfill writing assignments, and that LLM detection doesn't work, and that "watermarking" won't come to the rescue. There's lots of on-going published discussion, and even more discussion in real life.

As documented by the MLA-CCCC Joint Task Force on Writing and AI, the conclusion seems to be a combination of bringing AI explicitly into the class, and designing some assignments where students need to function without it.

In one recent example, Joe Moxley has posted the syllabus for his course "Writing with Artificial Intelligence – Syllabus (ENC 3370)".