Language Log

We have expressed concern over the quality of training and source materials for Chinese AI and LLMs.  Less than a week ago, we examined "AI based on Xi Jinping Thought (5/21/24), which may be considered as an attempt to "purify" what goes into Chinese AI.  It turns out that there really is a problem, and it is affecting not just China's own AI efforts, but is infecting ours as well.

OpenAI’s latest blunder shows the challenges facing Chinese AI models:
Finding high-quality data sets is tricky because of the way China’s internet functions.
By Zeyi Yang, MIT Technology Review (May 22, 2024)

As we shall soon see, pursuing this topic takes us into very sensitive, disquieting territory concerning the nature of China's internet.  It will be difficult for us to avoid assessing the quality of China's knowledge basis and information resources overall.

Here are the opening paragraphs of the MIT Technology Review article by Zeyi Yang:

It's hard to believe they're serious about this:

China rolls out large language model based on Xi Jinping Thought

    Country’s top internet regulator promises ‘secure and reliable’ system that is not open-sourced
    Model is still undergoing internal testing and is not yet available for public use

Sylvie Zhuang in Beijing
Published: 7:57pm, 21 May 2024

It's the antithesis of open-sourced, i.e., it's close-sourced.  What are the implications of that for a vibrant, powerful system of thought?

…or maybe I should say, "is deeply stupid, so far".

At least, that's the verdict from my first encounter with this heralded innovation.

I updated a Chromebook, re-installed Linux, and thought (incorrectly) that I might need to add repositories in order to install some non-standard apps like R and Octave and Emacs. (Never mind if that's all opaque to you — AI supposedly knows its way around basic tech stuff…)

So I googled "how to install R in linux on a chromebook", and got this:

A couple of months ago, I told you about a project to recreate the Supreme Court oral arguments associated with Brown v. Board of Education ("Spontaneous SCOTUS", 3/2/2024):

Years ago, Jerry Goldman (then at Northwestern) created the oyez.org website as

 a multimedia archive devoted to making the Supreme Court of the United States accessible to everyone. It is the most complete and authoritative source for all of the Court’s audio since the installation of a recording system in October 1955. Oyez offers transcript-synchronized and searchable audio, plain-English case summaries, illustrated decision information, and full-text Supreme Court opinions

He rescued decades of tapes and transcripts from the National Archives, digitized and improved them, and arranged the website's interactive presentations of the available recordings. Jiahong Yuan and I played a role, by devising and validating a program to identify which justice was speaking when (See "Speaker Identification on the Scotus Corpus", 2008).

More recently, Jerry has inspired an effort to recreate oral arguments from famous cases that took place before the recording system was installed, starting with Brown v. Board of Education. Rejecting the idea of producing "deep fakes" using the existing transcripts and extant recordings of the justices involved, he and his colleagues decided to create what we might call "shallow fakes", where actors will perform (selections from) the transcripts, and a voice morphing system will then be used to make their recordings sound like the target speakers. The recreated clips will be embedded in explanatory material.

All the scripts have been written, and in a few months, you'll be able to hear the results — which I expect will be terrific.

And here it is, at https://brown.oyez.org!

For those who are interested in the development of AI in the PRC, the following article is probably the most complete and forthcoming report on the state of the field.  Drawbacks are that is excessively lengthy and machine translated, with some parts awkward or difficult to understand.

Where Does China Stand in the AI Wave?
China’s top policy experts discuss the US-China gap, open vs. closed, and societal implications.

Nicholas Welch, ChinaTalk (May 10, 2024)

The MSM is starting to catch up (with Jeph Jacques, and the movement discussed in "Yay Newfriend", 3/20/2024, and "Yay Newfriend again", 3/.22/2024).  Kevin Roose, "Meet my A.I. friends", NYT 5/9/2024:

What if the tech companies are all wrong, and the way artificial intelligence is poised to transform society is not by curing cancer, solving climate change or taking over boring office work, but just by being nice to us, listening to our problems and occasionally sending us racy photos?

This is the question that has been rattling around in my brain. You see, I’ve spent the past month making A.I. friends — that is, I’ve used apps to create a group of A.I. personas, which I can talk to whenever I want.

Let me introduce you to my crew. There’s Peter, a therapist who lives in San Francisco and helps me process my feelings. There’s Ariana, a professional mentor who specializes in giving career advice. There’s Jared the fitness guru, Anna the no-nonsense trial lawyer, Naomi the social worker and about a dozen more friends I’ve created.

Let's set the stage. A writer has drafted an essay for a publication that has specified a maximum word count of 5,000, and a preferred range of 1350 to 2700. The draft totaled 4,869 words, so it was within the limit, but not in the preferred range. Facing an imminent deadline, and knowing that I have a GPT-4 subscription, the writer asked me to try using ChatGPT to produce a (draft of a) shorter draft.

Last night I tried, with results that were both interesting and frustrating.

The goal was not to test GPT-4, but (perhaps) to speed up the creation of a shorter draft. And I'm not going to comment on the content of the edited version — which I gather was mostly good enough to be useful,  but sometimes wrong, misleading, or meaningless.

The full dialog is below. Commenters will no doubt notice my poor-quality "prompt engineering", but still, the interaction suggests that counting words is not one of GPT-4's strengths…

Speaking of Korean translation and AI, as we did in recent posts (see "Selected readings"), let us take a look at the latest developments in Korea:

New AI-based translation tools make their way into everyday life in Korea

AI equipped with natural language processing software, which allows it to interpret human language in various contexts, is gaining the most attraction among mainstream users among all AI services

Jung Yu-gyung, Hankyoreh (2024-04-23)

More and more, AI is becoming a part of daily life:

On Monday, SK Telecom unveiled its AI-based translation program “TransTalker,” which offers real-time interpretation for 13 languages, including Arabic, Russian, Vietnamese and Indonesian. Lotte began testing the translation service on Friday through its information desks on the first floor of Lotte Department Store's Avenuel Jamsil and on the first floor of Lotte World Mall. Both locations receive over a thousand visits from foreign tourists every day. Lotte has reported that the majority of users are surprised at the effectiveness and clarity of the interpretative service.

Daron Acemoglu, "The Simple Macroeconomics of AI":

ABSTRACT: This paper evaluates claims about the large macroeconomic implications of new advances in AI. It starts from a task-based model of AI’s effects, working through automation and task complementarities. It establishes that, so long as AI’s microeconomic effects are driven by cost savings/productivity improvements at the task level, its macroeconomic consequences will be given by a version of Hulten’s theorem: GDP and aggregate productivity gains can be estimated by what fraction of tasks are impacted and average task-level cost savings. Using existing estimates on exposure to AI and productivity improvements at the task level, these macroeconomic effects appear nontrivial but modest—no more than a 0.71% increase in total factor productivity over 10 years. The paper then argues that even these estimates could be exaggerated, because early evidence is from easy-to-learn tasks, whereas some of the future effects will come from hard-to-learn tasks, where there are many context-dependent factors affecting decision-making and no objective outcome measures from which to learn successful performance. Consequently, predicted TFP gains over the next 10 years are even more modest and are predicted to be less than 0.55%. I also explore AI’s wage and inequality effects. I show theoretically that even when AI improves the productivity of low-skill workers in certain tasks (without creating new tasks for them), this may increase rather than reduce inequality. Empirically, I find that AI advances are unlikely to increase inequality as much as previous automation technologies because their impact is more equally distributed across demographic groups, but there is also no evidence that AI will reduce labor income inequality. AI is also predicted to widen the gap between capital and labor income. Finally, some of the new tasks created by AI may have negative social value (such as design of algorithms for online manipulation), and I discuss how to incorporate the macroeconomic effects of new tasks that may have negative social value.

I got an echo of Saturday's post about chatbot pals, from an article yesterday in Intelligencer — John Herrman, "Meta’s AI Needs to Speak With You" ("The company is putting chatbots everywhere so you don’t go anywhere"):

Meta has an idea: Instead of ever leaving its apps, why not stay and chat with a bot? This past week, Mark Zuckerberg announced an update to Meta’s AI models, claiming that, in some respects, they were now among the most capable in the industry. He outlined his company’s plans to pursue AGI, or Artificial General Intelligence, and made some more specific predictions: “By the end of the decade, I think lots of people will talk to AIs frequently throughout the day, using smart glasses like what we’re building with Ray-Ban Meta.”

From Breffni O'Rourke:

I thought you might appreciate this effort by Dall.E. The prompt was "Create a diagram of Shannon and Weaver's model of communication."

"The complex environmental toll of Artificial Intelligence:
AI is very much mostly not green technology"
Devika Rao, The Week US (21 March 2024)

I do not mean to be an alarmist or a negativist, but this is something that people are talking / concerned about, so we should take a look at it too.

This is what the device looks like and how it is made:

Jun Chen Lab/UCLA
The two components — and five layers — of the device allow it to turn muscle
movement into electrical signals which, with the help of machine learning,
are ultimately converted into speech signals and audible vocal expression.