{"id":67118,"date":"2024-11-28T07:45:23","date_gmt":"2024-11-28T12:45:23","guid":{"rendered":"https:\/\/languagelog.ldc.upenn.edu\/nll\/?p=67118"},"modified":"2024-12-06T23:15:23","modified_gmt":"2024-12-07T04:15:23","slug":"searles-chinese-room-and-the-enigma-of-understanding","status":"publish","type":"post","link":"https:\/\/languagelog.ldc.upenn.edu\/nll\/?p=67118","title":{"rendered":"Searle's \"Chinese room\" and the enigma of understanding"},"content":{"rendered":"

In this comment<\/a> to \"'Neutrino Evidence Revisited (AI Debates)' | Is Mozart's K297b authentic?<\/a>\" (11\/13\/24), I questioned whether John Searle<\/a>'s \"Chinese room<\/a>\" argument was intelligently designed and encouraged those who encounter it to reflect on what it did \u2014 and did not \u2014 demonstrate.<\/p>\r\n

In the same comment, I also queried the meaning of \"understand\" and its synonyms (\"comprehend\", and so forth).<\/p>\r\n

Both the \"Chinese room\" and \"understanding\" had been raised by skeptics of AI, so here I'm treating them together.<\/p>\r\n

<\/p>\r\n

I will say flat out that I don't think the Chinese room argument proved anything useful or conclusive with regard to AI.\u00a0 I could talk at much greater length about the weaknesses of the Chinese room, but — in the interest of efficiency — I will simply point out one fatal flaw (or rather a complex of weaknesses that amount to a fatal flaw) in its construction.<\/p>\r\n

Here's the Chinese room argument in a nutshell:<\/p>\r\n

The Chinese room argument<\/b> holds that a computer executing a program<\/a> cannot have a mind<\/a>, understanding<\/a>, or consciousness<\/a>, regardless of how intelligently or human-like the program may make the computer behave. The argument was presented in a 1980 paper by the philosopher John Searle<\/a> entitled \"Minds, Brains, and Programs\" and published in the journal Behavioral and Brain Sciences<\/a><\/i>. Before Searle, similar arguments had been presented by figures including Gottfried Wilhelm Leibniz<\/a> (1714), Anatoly Dneprov<\/a> (1961), Lawrence Davis (1974) and Ned Block<\/a> (1978). Searle's version has been widely discussed in the years since. The centerpiece of Searle's argument is a thought experiment<\/a> known as the Chinese room<\/b>.<\/p>\r\n

The thought experiment starts by placing a computer that can perfectly converse in Chinese in one room, and a human that only knows English in another, with a door separating them. Chinese characters are written and placed on a piece of paper underneath the door, and the computer can reply fluently, slipping the reply underneath the door. The human is then given English instructions which replicate the instructions and function of the computer program to converse in Chinese. The human follows the instructions and the two rooms can perfectly communicate in Chinese, but the human still does not actually understand the characters, merely following instructions to converse. Searle states that both the computer and human are doing identical tasks, following instructions without truly understanding or \"thinking\".<\/p>\r\n

The argument is directed against the philosophical positions of functionalism<\/a> and computationalism<\/a>, which hold that the mind may be viewed as an information-processing system operating on formal symbols, and that simulation of a given mental state is sufficient for its presence. Specifically, the argument is intended to refute a position Searle calls the strong AI hypothesis<\/b>: \"The appropriately programmed computer with the right inputs and outputs would thereby have a mind in exactly the same sense human beings have minds.\"<\/p>\r\n

Although its proponents originally presented the argument in reaction to statements of artificial intelligence<\/a> (AI) researchers, it is not an argument against the goals of mainstream AI research because it does not show a limit in the amount of intelligent behavior a machine can display. The argument applies only to digital computers running programs and does not apply to machines in general. While widely discussed, the argument has been subject to significant criticism and remains controversial among philosophers of mind<\/a> and AI researchers.<\/sup><\/p>\r\n

(Wikipedia<\/a>)<\/p>\r\n

Fans \/ devotees of Searle will not be happy with what I say in the following paragraphs.\u00a0 He makes far too many assumptions for comfort, and I do not believe that we — the recipients of his argument — should be obliged to smooth over his unexplained assumptions.\u00a0 A surfeit of unexplained assumptions amounts to an abandonment, rejection, or negation.<\/p>\r\n

I will not point out every defect in the Chinese room argument, but will merely signal a series of unexplained \/ inexplicable terms \/ expressions that do not contribute to the goal of Searle's contention.\u00a0 Whether or not Searle used all of these exact terms, I trust Wikipedia enough to believe that they convey the gist of Searle's intent.<\/p>\r\n

For a much more exhaustive, philosophically professional account of Searle's experiment with extensive documentation and quotations, see Larry Hauser, \"Chinese Room Argument<\/a>\", Internet Encyclopedia of Philosophy, also David Cole, \"The Chinese Room Argument<\/a>\", Stanford Encyclopedia of Philosophy (First published Fri Mar 19, 2004; substantive revision Wed Oct 23, 2024<\/em>).<\/p>\r\n

converse<\/b> — This generally signifies exchanging ideas, information, etc. through talking.\u00a0 Who \/ What taught the computer to talk?\u00a0 How does it talk?\u00a0 By writing \/ typing out its statements and answers?\u00a0 By listening to the human's questions and answers?\u00a0 But don't forget that the door is there to prevent that from happening, except for the writing on the slips of paper passed beneath the door (which defeats the purpose of the door).<\/p>\r\n

perfectly<\/b> — Not only does the computer allegedly speak Chinese, it does so perfectly<\/b>.\u00a0 How did it gain this mastery?<\/p>\r\n

in Chinese; only in English<\/b> — Somebody or something has to translate between the two, but Searle completely leaves that essential step out; this is where I almost stopped engaging with Searle's defective reasoning and had to force myself to continue.<\/p>\r\n

door<\/b> — Who put it between the Chinese speaking computer and the English speaking human?\u00a0 That's an arbitrary act without an actor.\u00a0 Moreover, the door is ostensibly meant to separate the computer and the human, but then Searle cheats by having an opening at the bottom of the door through which it is easy to slip pieces of paper with Chinese characters written on them.<\/p>\r\n

speaking and writing<\/b> — In Searle's experiment, there is actually no speaking going on, only writing.\u00a0 Who does the writing?\u00a0 Who is slipping those pieces of paper under the door?\u00a0 How did the computer and the human become literate in the written Chinese that is being slipped under the door on pieces of paper?\u00a0 Remember that full \/ \"perfect\" literacy is a difficult task, whether for a computer or a human.\u00a0 Especially with maddeningly complex sinographs.\u00a0 Remember \u60e1\u60e1<\/a>, for just a tiny taste?<\/p>\r\n

Enough about the fundamental mechanics of the Chinese room experiment.\u00a0 I could easily point out many more defects in Searle's argument, not least the fact that it is about two rooms<\/b>, a Chinese room and an English room, not just a Chinese room.\u00a0 Something else that irks me no end, namely, why Chinese?\u00a0 Why not Russian or German or Maori or Cantonese or Cia-Cia (in which script — Latin or Arabic or Hangul?), Dungan (in which script — Arabic, Cyrillic, Sinographic?), Nakhi \/ Nashi \/ Nakhi?\u00a0<\/p>\r\n

\"Chinese\" (Mandarin or Shanghainese or Minnan, or…).\u00a0 I think Searle chose \"Chinese\" (i.e., Chinese characters), not one of the possible spoken Sinitic languages or topolects or dialects, because of its propensity for mystification, exoticization, and obfuscation.\u00a0 Does Searle's computer understand<\/b> so much as a single Chinese character, much less the 10,000 that it would need to know <\/b>how to \"converse\" with the human on the other side of the door?<\/p>\r\n

In my estimation, Searle's so-called \"Chinese room argument\" is nothing but a complicated and improbable form of the more reasonable and workable Turing Test, for which see this abbreviated, straightforward account:<\/p>\r\n

The Turing Test is a method of inquiry in artificial intelligence (AI<\/a>) for determining whether or not a computer is capable of thinking like a human being. The test is named after Alan Turing, the founder of the Turing Test and an English computer scientist, cryptanalyst, mathematician and theoretical biologist.<\/p>\r\n

Turing proposed that a computer can be said to possess artificial intelligence if it can mimic human responses under specific conditions. The original Turing Test requires three terminals, each of which is physically separated from the other two. One terminal is operated by a computer, while the other two are operated by humans.<\/p>\r\n

During the test, one of the humans functions as the questioner, while the second human and the computer function as respondents. The questioner interrogates the respondents within a specific subject area, using a specified format and context. After a preset length of time or number of questions, the questioner is then asked to decide which respondent was human and which was a computer.<\/p>\r\n

The test is repeated many times. If the questioner makes the correct determination in half of the test runs or less, the computer is considered to have artificial intelligence because the questioner regards it as \"just as human\" as the human respondent.<\/p>\r\n

—–<\/p>\r\n

\"What is the Turing Test?<\/a>\" by Benjamin St. George and Alexander S. Gillis, Tech Target (updated in August 2024)<\/p>\r\n

As for \"understanding\", I will note only that Searle's \"Chinese room\" — with a door separating the human and the computer and an opening beneath it to permit the passage of communication between the two — does demonstrate an awareness of the meaning of the significance of that process.<\/p>\r\n

From Middle English<\/a><\/span> understanden<\/a><\/i>, from Old English<\/a><\/span> understandan<\/a><\/i> (<\/span>\u201c<\/span>to understand<\/span>\u201d<\/span>)<\/span>, from Proto-West Germanic<\/a><\/span> *understandan<\/a><\/i> (<\/span>\u201c<\/span>to stand between, understand<\/span>\u201d<\/span>)<\/span>, from Proto-Germanic<\/a><\/span> *understandan\u0105<\/a><\/i> (<\/span>\u201c<\/span>to stand between, understand<\/span>\u201d<\/span>)<\/span>, equivalent to Old English<\/a><\/span> under-<\/a><\/i> (<\/span>\u201c<\/span>between, inter-<\/span>\u201d<\/span>)<\/span> + standan<\/a><\/i> (<\/span>\u201c<\/span>to stand<\/span>\u201d<\/span>)<\/span> (Modern English under-<\/a><\/i> +\u200e stand<\/a><\/i>). Cognate with Old Frisian<\/a><\/span> understonda<\/a><\/i> (<\/span>\u201c<\/span>to understand, experience, learn<\/span>\u201d<\/span>)<\/span>, Old High German<\/a><\/span> understantan<\/a><\/i> (<\/span>\u201c<\/span>to understand<\/span>\u201d<\/span>)<\/span>, Middle Danish<\/a><\/span> understande<\/a><\/i> (<\/span>\u201c<\/span>to understand<\/span>\u201d<\/span>)<\/span>. Compare also Saterland Frisian<\/a><\/span> understunda<\/a><\/i>, unnerstounde<\/a><\/i> (<\/span>\u201c<\/span>to dare, survey, measure<\/span>\u201d<\/span>)<\/span>, Dutch<\/a><\/span> onderstaan<\/a><\/i> (<\/span>\u201c<\/span>to undertake, presume<\/span>\u201d<\/span>)<\/span>, German<\/a><\/span> unterstehen<\/a><\/i> (<\/span>\u201c<\/span>to be subordinate<\/span>\u201d<\/span>)<\/span>.<\/p>\r\n

(Wiktionary<\/a>)<\/p>\r\n

Old English understandan<\/span> \"comprehend, grasp the idea of, achieve comprehension; receive from a word or words or from a sign or symbol the idea it is intended to convey;\" also \"view in a certain way,\" probably literally \"stand in the midst of,\" from under<\/a> + standan<\/span> \"to stand\" (see stand<\/a> (v.)).<\/p>\r\n

If this is the meaning, the under<\/span> is not the usual word meaning \"beneath,\" but from Old English under<\/span>, from PIE *nter-<\/span> \"between, among\" (source also of Sanskrit antar<\/span> \"among, between,\" Latin inter<\/span> \"between, among,\" Greek entera<\/span> \"intestines;\" see inter-<\/a>). Related: Understood<\/span>; understanding<\/a>.<\/p>\r\n

That is the suggestion in Barnhart, but other sources regard the \"among, between, before, in the presence of\" sense of Old English prefix and preposition under<\/span> as other meanings of the same word. \"Among\" seems to be the sense in many Old English compounds that resemble understand<\/em><\/span>, such as underfinden<\/span> \"be aware, perceiver\" (c. 1200); undersecan<\/span> \"examine, investigate, scrutinize\" (literally \"underseek\"); under\u00f0encan<\/span> \"consider, change one's mind;\" underginnan<\/span> \"to begin;\" underniman<\/span> \"receive.\" Also compare undertake<\/a>, which in Middle English also meant \"accept, understand<\/em>.\"<\/p>\r\n

It also seems to be the sense still in expressions such as under such circumstances<\/span>. Perhaps the ultimate sense is \"be close to;\" compare Greek epistamai<\/span> \"I know how, I know,\" literally \"I stand upon.\"<\/p>\r\n

Similar formations are found in Old Frisian (understonda<\/span>), Middle Danish (understande<\/span>), while other Germanic languages use compounds meaning \"stand before\" (German verstehen<\/span>, represented in Old English by forstanden<\/span> \"understand,\" also \"oppose, withstand\"). For this concept, most Indo-European languages use figurative extensions of compounds that literally mean \"put together,\" or \"separate,\" or \"take, grasp\" (see comprehend<\/a>).<\/p>\r\n

The range of spellings of understand<\/em><\/span> in Middle English (Middle English Compendium lists 70, including understont, understounde, un\u00feurstonde, onderstonde, hunderstonde, oundyrston, wonderstande, urdenstonden<\/span>) perhaps reflects early confusion over the elements of the compound. Old English oferstandan<\/span>, Middle English overstonden<\/span>, literally \"over-stand\" seem to have been used only in literal senses.<\/p>\r\n

By mid-14c. as \"to take as meant or implied (though not expressed); imply; infer; assume; take for granted.\" The intransitive sense of \"have the use of the intellectual faculties; be an intelligent and conscious being\" also is in late Old English.<\/p>\r\n

In Middle English also \"reflect, muse, be thoughtful; imagine; be suspicious of; pay attention, take note; strive for; plan, intend; conceive (a child).\" In the Trinity Homilies (c. 1200), a description of Christ becoming human was that he understood mannish<\/span>.<\/p>\r\n

Also sometimes literal, \"to occupy space at a lower level\" (late 14c.) and, figuratively, \"to submit.\" For \"stand under\" in a physical sense, Old English had undergestandan<\/span>.<\/p>\r\n

(Etymonline)<\/p>\r\n

In conclusion, I quote the Stanford cognitive scientist and computer scientist, John McCarthy (from an article on his website)<\/a>:<\/p>\r\n

The Chinese Room Argument can be refuted in one sentence:<\/p>\r\n

Searle confuses the mental qualities of one computational process, himself for example, with those of another process that the first process might be interpreting, a process that understands Chinese, for example.<\/b><\/p>\r\n

'Nuff said, but McCarthy , being a sort of philosopher himself, also presents \"the argument in more detail\" and \"the refutation in still more detail\".\u00a0 He also explains what is required for a Chinese room that passes the Turing test, and, having \"developed time-sharing, invented LISP, and founded the field of Artificial Intelligence<\/a>\", sensibly emphasizes the role of translation between computer and human, briefly taking on the formidable Willard Van Orman Quine with regard to\u00a0\"the indeterminacy of radical translation\".<\/p>\r\n

Searle's idea against the notion of artificial consciousness may have been correct, but his Chinese room experiment did not serve to advance his cause.<\/p>\r\n

 <\/p>\r\n

Selected readings<\/b><\/p>\r\n