Chinese Room Argument (1980) › The Solution: The Chinese Room Thought Experiment
History of Language AIAll chapters Chinese Room Argument (1980) › The Solution: The Chinese Room Thought ExperimentOne section · Sections in this chapter3 / 9 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Searle's Chinese Room argument used a thought experiment to demonstrate that syntax alone cannot produce semantics, that symbol manipulation cannot produce understanding. The argument proceeded in several steps, each carefully designed to isolate the relevant issues and show why computation couldn't generate understanding.
Imagine a person locked in a room. This person understands no Chinese whatsoever. She receives sheets of paper with Chinese characters written on them, which she doesn't recognize or understand. She also has a large book of rules, written in English, that tells her how to manipulate these Chinese symbols. The rules might say things like "When you see the character pattern X followed by pattern Y, look up the appropriate response from section Z of the book, and write those characters on a new sheet." The person follows these rules mechanically, looking up patterns, finding corresponding responses, and producing Chinese character strings as output.
Now, suppose that the book of rules is detailed enough for the Chinese character strings produced as output to be indistinguishable from those a native Chinese speaker would produce. When given questions in Chinese, the room produces answers in Chinese that are appropriate and correct. To someone outside the room who speaks Chinese, the room appears to understand Chinese perfectly. They send in a question, get back an appropriate answer, and have no way of knowing that the person inside understands nothing.
But from inside the room, there is no understanding of Chinese at all. The person is just following rules, manipulating symbols based on their formal properties. She has no idea what the symbols mean, no connection to what they represent, no understanding of the Chinese language. She's processing syntax, pure symbol manipulation, without any semantics, without any meaning.
Searle then applied the analogy to computers: if the person in the room doesn't understand Chinese, and the room (person plus book of rules) doesn't understand Chinese, then why should we think that a computer running a program understands anything? A computer is just another system for manipulating symbols according to rules. The computer program is the book of rules. The computer's processing is the person following those rules. If the Chinese Room shows that rule-following cannot produce understanding, then computer programs, which are just sophisticated rule-following systems, cannot produce understanding either.
The argument applied to any AI system, not just language processing. If a system's behavior resulted from symbol manipulation according to rules, Searle argued that its sophistication did not amount to understanding. The system manipulated symbols, and the complexity of that rule-following created only the appearance of understanding.
One immediate response to Searle's argument was the "system reply": while the person in the room doesn't understand Chinese, the room as a whole, including the person, the book of rules, and the paper, does understand Chinese. Similarly, while individual components of a computer might not understand, the system as a whole does. Searle responded by imagining that the person memorized the book of rules and did all the symbol manipulation in her head. Now there's no system beyond the person, yet she still doesn't understand Chinese. The system reply, he argued, simply begs the question by assuming that the system understands without explaining how or why.
Searle distinguished between two kinds of intentionality, two ways that mental states can be about things. Original intentionality is the kind that minds have naturally. Our beliefs, desires, and thoughts are about things in the world. This intentionality is intrinsic to mental states. Derived intentionality is the kind that symbols, words, and representations have. The word "cat" is about cats, but only because we interpret it that way. The intentionality of symbols is derived from the original intentionality of minds that use them.
Computers and their programs have only derived intentionality. The symbols they manipulate are about things only because we interpret them that way. When a computer processes "cat," it manipulates a symbol whose reference comes from our interpretation. That interpretation comes from us, not from the computer. The computer has no original intentionality or intrinsic aboutness and therefore no understanding of its own.
The Chinese Room showed that no amount of symbol manipulation, no matter how sophisticated, could produce original intentionality. The person in the room was manipulating symbols according to rules, but those symbols meant nothing to her. They were just formal patterns. Similarly, computers manipulate symbols according to programs, but those symbols mean nothing to the computer. They're just formal patterns. Understanding requires original intentionality, and computation provides only syntax manipulation, which cannot produce semantics or understanding.