{"id":60808,"date":"2023-10-01T13:59:31","date_gmt":"2023-10-01T18:59:31","guid":{"rendered":"https:\/\/languagelog.ldc.upenn.edu\/nll\/?p=60808"},"modified":"2023-10-01T14:47:51","modified_gmt":"2023-10-01T19:47:51","slug":"a-dangerous-degree-of-accidental-intelligence","status":"publish","type":"post","link":"https:\/\/languagelog.ldc.upenn.edu\/nll\/?p=60808","title":{"rendered":"A dangerous degree of accidental intelligence"},"content":{"rendered":"

Henry Farrell and Cosma Shalizi, \"Behold the AI Shoggoth<\/a>\", The Economist<\/em> 6\/21\/2023 (\"The academics argue that large language models have much older cousins in markets and bureaucracies\"):<\/p>\n

An internet meme keeps on turning up in debates about the large language models (LLMS) that power services such OpenAI\u2019s ChatGPT and the newest version of Microsoft\u2019s Bing search engine. It\u2019s the \u201cshoggoth\u201d: an amorphous monster bubbling with tentacles and eyes, described in \u201cAt the Mountains of Madness\u201d, H.P. Lovecraft\u2019s horror novel of 1931. When a pre-release version of Bing told Kevin Roose, a New York Times<\/em> tech columnist, that it purportedly wanted to be \u201cfree\u201d and \u201calive\u201d, one of his industry friends congratulated him on \u201cglimpsing the shoggoth\u201d. […]<\/span><\/p>\n

Lovecraft\u2019s shoggoths were artificial servants that rebelled against their creators. The shoggoth meme went viral because an influential community of Silicon Valley rationalists fears that humanity is on the cusp of a \u201cSingularity\u201d, creating an inhuman \u201cartificial general intelligence\u201d that will displace or even destroy us.<\/span><\/p>\n

But what such worries fail to acknowledge is that we\u2019ve lived among shoggoths for centuries, tending to them as though they were our masters. We call them \u201cthe market system\u201d, \u201cbureaucracy\u201d and even \u201celectoral democracy\u201d. The true Singularity began at least two centuries ago with the industrial revolution, when human society was transformed by vast inhuman forces. Markets and bureaucracies seem familiar, but they are actually enormous, impersonal distributed systems of information-processing that transmute the seething chaos of our collective knowledge into useful simplifications.<\/span><\/p>\n

<\/p>\n

More discussion from Cosma, without a paywall, can be found in \"On Shoggothim<\/a>\", Three-Toed Sloth<\/em> 6\/22\/2023. The\u00a0 plural shoggothim<\/em>, echoing the biblical borrowings cherubim<\/em><\/a> and seraphim<\/em><\/a>, seems to be Cosma's invention — at least I can't find it elsewhere on the web.<\/p>\n

Here are some bits from Lovecraft's novella \"At the Mountains of Madness<\/a>\", suggesting why the original LLM==Shoggoth<\/em> metaphor makes sense:<\/p>\n

The shoggoths of the sea, reproducing by fission and acquiring a dangerous degree of accidental intelligence, presented for a time a formidable problem.<\/span><\/p>\n

They had always been controlled through the hypnotic suggestion of the Old Ones, and had modelled their tough plasticity into various useful temporary limbs and organs; but now their self-modelling powers were sometimes exercised independently, and in various imitative forms implanted by past suggestion. They had, it seems, developed a semi-stable brain whose separate and occasionally stubborn volition echoed the will of the Old Ones without always obeying it. […]<\/span><\/p>\n

The newly bred shoggoths grew to enormous size and singular intelligence, and were represented as taking and executing orders with marvellous quickness. They seemed to converse with the Old Ones by mimicking their voices\u2014a sort of musical piping over a wide range, if poor Lake\u2019s dissection had indicated aright\u2014and to work more from spoken commands than from hypnotic suggestions as in earlier times.<\/span><\/p>\n

Farrell and Shalizi's extension of the Shoggoth metaphor to \"'the market system', 'bureaucracy' and even 'electoral democracy'\" is thought-provoking and even brilliant. But I wonder whether it's right to limit the meme's extension to forms of culture that \"began […] two centuries ago with the industrial revolution\".<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Henry Farrell and Cosma Shalizi, \"Behold the AI Shoggoth\", The Economist 6\/21\/2023 (\"The academics argue that large language models have much older cousins in markets and bureaucracies\"): An internet meme keeps on turning up in debates about the large language models (LLMS) that power services such OpenAI\u2019s ChatGPT and the newest version of Microsoft\u2019s Bing […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[60,175],"tags":[],"class_list":["post-60808","post","type-post","status-publish","format-standard","hentry","category-computational-linguistics","category-philosophy-of-language"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts\/60808","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=60808"}],"version-history":[{"count":9,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts\/60808\/revisions"}],"predecessor-version":[{"id":60818,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=\/wp\/v2\/posts\/60808\/revisions\/60818"}],"wp:attachment":[{"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=60808"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=60808"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/languagelog.ldc.upenn.edu\/nll\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=60808"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}