Language Log

« previous post | next post »

With this post, we are already acquainted with Inspur's Yuan 1.0, "one of the most advanced deep learning language models that can generate coherent Chinese texts."  Now, with the present article, we will delve more deeply into the potentials and pitfalls of Inspur's deep learning language model:

"Inspur unveils GPT-3 equivalent for Chinese language", by Wei Sheng, TechNode (1026/21)

The model is trained with 245.7 billion parameters—the number of weights in an artificial neural network, according to the company. This is more than the Elon Musk-backed GPT-3 language model for English, which has 175 billion parameters. Inspur said the Yuan model was trained with 5 terabytes of datasets.

…

Chinese server maker Inspur on Tuesday released Yuan 1.0, one of the most advanced deep learning language models that can generate coherent Chinese texts. 

The model is trained with 245.7 billion parameters—the number of weights in an artificial neural network, according to the company. This is more than the Elon Musk-backed GPT-3 language model for English, which has 175 billion parameters. Inspur said the Yuan model was trained with 5 terabytes of datasets.

The release of Yuan is a milestone for the Chinese natural language processing (NLP) industry. NLP is an important branch of artificial intelligence (AI) and a backbone for computers to understand human language.

Inspur said in a statement that its AI Research Institute had to develop “a unique development approach compared to English” to train the model, including addressing challenges such as “the lack of a prior high-quality Chinese-language corpus.” A corpus is a collection of texts used to train language models.

Inspur said the language model is “extremely adept” at natural language generation (NLG) tasks—the processing of generating natural language text using computers. The company said in a paper published in October that “only less than 50% of the time” human testers could distinguish between text generated by the model and human-written ones.

…

In May 2021, a group of US researchers at Georgetown University found powerful NLG tools can be used to fuel disinformation. Inspur acknowledges such a possibility and vows to oversee the appropriate use of the model.  

“Since the model can generate articles that are difficult to detect whether they are human written or not, the risk of misuse becomes higher,” an Inspur spokesperson told TechNode on Tuesday. “Therefore we need to regulate the application of the model in the future.”

…

Jinan-based Inspur is the world’s third-largest maker of servers, according to market research firm IDC. The 72-year-old state-owned company used to be a manufacturer of electronic devices. It first entered the server market in 1995 as a partner of US chipmaker Intel. 

In June 2020, the US Defence Department added Inspur, alongside 19 other Chinese companies, to a list that it deemed to be owned or controlled by the Chinese military, paving the way for further financial or export sanctions. In July 2020, Intel reportedly restored shipment to the company.

It seems that malevolent actors could potentially use such powerful deep learning language models for nefarious purposes.  Of course, one can use already existing computer hardware and software for nefarious purposes.  But deep learning language models are poised to enter another realm, one in which the computer itself can start thinking like a human being.  That becomes very tricky and very scary.

Remember Hal 9000?

 

Selected readings

• "The implications of Chinese for AI development" (10/25/21) — with extensive bibliographical references
• "Lively Blind Men" (10/26/21)

 

[Thanks to Odette Yang]

October 27, 2021 @ 9:27 am · Filed by Victor Mair under Artificial intelligence, Language and computers

Permalink