# AI Doesn't REALLY Understand. We've Said That Before. ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before.webp]] **Creator:** Count Grey Matter · **Published:** 2026-10-18 · **Length:** 22:58 · [Watch on YouTube](https://www.youtube.com/watch?v=7O84mdxehWY) > Caption source: YouTube **auto-generated English** captions (`en`), downloaded with **yt-dlp** using the **android** player client after an initial HTTP 429 on the default client (~25 s retry). The track covers the whole video (0:00–22:58) with no gaps; no youtube-transcript-api or TubeAlfred credits were used. YouTube chapters are used as section headings. Screenshots come from the downloadable progressive MP4 stream (format 18, ~360p). Host persona: **the Count** (Count Grey Matter), with a lodger named after **ELIZA**. Clips of **Geoffrey Hinton** are from his IASEAI '25 talk "What Is Understanding?" (timestamps cited in the description: 7:52, 10:25, 17:31). Sources listed in the description (RAND, Nature, ACL, arXiv, NYT, etc.) were used for names, dates and paper titles; no separate web cross-check beyond that description list. ## Outline ### 0:00 Cold open - [0:00](https://www.youtube.com/watch?v=7O84mdxehWY&t=0s) Cold open on **Hubert Dreyfus**'s 1965 RAND report *Alchemy and Artificial Intelligence*: chess programs "couldn't yet beat even an amateur." - [0:13](https://www.youtube.com/watch?v=7O84mdxehWY&t=13s) Two years later at MIT he sat down against one (**Mac Hack VI**) and **it beat him**. He spent the next 50 years arguing the machines still weren't thinking—"and on one point, he was right." - [0:31](https://www.youtube.com/watch?v=7O84mdxehWY&t=31s) "I've watched people draw a line between minds and machines for a very long time. Tonight, I'd like to show you every place it has been." - [0:44](https://www.youtube.com/watch?v=7O84mdxehWY&t=44s) Title card / "Good evening." ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0003.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0020.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0045.webp]] ### 0:51 Part One: The Line - [0:54](https://www.youtube.com/watch?v=7O84mdxehWY&t=54s) **Part one, the line.** Some jobs don't care what they run on: flight runs on feathers or aluminum and nobody says the airplane is pretending. Digestion needs a gut. "So, which kind of job is thinking?" - [2:01](https://www.youtube.com/watch?v=7O84mdxehWY&t=121s) For a few hundred years people answered by drawing a line: machine-side vs mind-side. - [2:10](https://www.youtube.com/watch?v=7O84mdxehWY&t=130s) Oldest remembered line: **arithmetic**. In **1642** an 18-year-old in Rouen, **Blaise Pascal**, built a brass-wheel machine (the **Pascaline**) to spare his tax-official father endless sums. - [2:27](https://www.youtube.com/watch?v=7O84mdxehWY&t=147s) Pascal (Pensées): the arithmetical machine "produces effects which approach nearer to thought than all the actions of animals," but does nothing that would let us attribute **will** to it. He granted the effect and kept back the someone. "Remember that move. Everyone makes it." - [2:20](https://www.youtube.com/watch?v=7O84mdxehWY&t=140s) Within generations, nobody thought sums were thinking. **"Computer" used to be a job title** for a person; then the word moved into the machine and took the sums with it. - [2:42](https://www.youtube.com/watch?v=7O84mdxehWY&t=162s) One word to listen for: **just**. "It's just calculation. It's just patterns. It's just auto complete." ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0060.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0090.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0120.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0150.webp]] ### 2:53 Part Two: Calculation - [2:55](https://www.youtube.com/watch?v=7O84mdxehWY&t=175s) **Part two, calculation.** Can a machine play chess? For a long time chess was the line. - [3:05](https://www.youtube.com/watch?v=7O84mdxehWY&t=185s) In **1957** **Herbert Simon** predicted a digital computer would be world chess champion within ten years. "It took **40**." - [3:27](https://www.youtube.com/watch?v=7O84mdxehWY&t=207s) **May 1997**: **Garry Kasparov** vs **Deep Blue**. ~**200 million** positions/second, grandmaster-written scoring formula, look-ahead. Deep Blue won the match; in one game it played so subtly Kasparov suspected human help. It had already won a single game the year before. - [3:56](https://www.youtube.com/watch?v=7O84mdxehWY&t=236s) **Douglas Hofstadter**, who had said only a general mind could beat everyone at chess, after that game: "My god, I used to think chess required thought. Now, I realize it doesn't." Chess was moved to another shelf. ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0180.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0220.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0260.webp]] ### 4:18 Part Three: Intuition - [4:21](https://www.youtube.com/watch?v=7O84mdxehWY&t=261s) **Part three, intuition.** Can a machine have intuition? - [4:24](https://www.youtube.com/watch?v=7O84mdxehWY&t=264s) Same year Deep Blue won, Princeton astrophysicist **Piet Hut** told the *New York Times* it may be a hundred years before a computer beats humans at **Go**. "It took **19**." - [4:47](https://www.youtube.com/watch?v=7O84mdxehWY&t=287s) Go: 19×19 grid; rules fit on a postcard; ~**10^170** legal positions (vs ~10^80 atoms in the visible universe). Deep Blue's method drowns: too many moves and no writable scoring rule. Masters call good shape "intuition." - [5:27](https://www.youtube.com/watch?v=7O84mdxehWY&t=327s) AlphaGo's three pieces: a **policy** net with a hunch about promising moves (from human games then self-play); a **value** net with a hunch about who is winning; **search** that plays out imagined games from promising moves. "Nobody wrote its strategy. It learned its hunches." - [6:04](https://www.youtube.com/watch?v=7O84mdxehWY&t=364s) **Seoul, March 2016**: AlphaGo vs **Lee Sedol**. Game 2 **move 37** — "It's not a human move… So beautiful" (European champion). Lee won one of five with a "divine" move; the machine won four. People said "just pattern matching." "It is. So is a master's intuition. 10,000 games folded into a hunch." ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0300.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0340.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0380.webp]] ### 6:47 Part Four: Language - [6:49](https://www.youtube.com/watch?v=7O84mdxehWY&t=409s) **Part four, language.** Can a machine hold a conversation? **Alan Turing**'s test: judge chats with person and machine and must say which is which. - [7:03](https://www.youtube.com/watch?v=7O84mdxehWY&t=423s) Fewer remember what it answered. In **1949** Manchester surgeon **Geoffrey Jefferson** (Lister Oration): not until a machine can write a sonnet or compose a concerto "because of thoughts and emotions felt," and know that it had written it, could we agree machine equals brain. "Hold on to that surgeon." - [7:39](https://www.youtube.com/watch?v=7O84mdxehWY&t=459s) Turing's dry reply: if the only way to be sure a machine thinks is to *be* it, then the only way to be sure anyone thinks is to be them—and nobody lives like that. Instead, "the polite convention that everyone thinks." - [7:55](https://www.youtube.com/watch?v=7O84mdxehWY&t=475s) Turing predicted that within ~50 years a machine would fool an average judge ~**30**/100 after five minutes. - [8:08](https://www.youtube.com/watch?v=7O84mdxehWY&t=488s) In **2025**, **Cameron R. Jones** and **Benjamin K. Bergen** (San Diego) ran the game properly. Best model, told to play a shy young person fluent in internet slang, was picked as human **73**/100—more often than actual humans. Turing asked for 30; he got 73, a quarter-century late. - [8:22](https://www.youtube.com/watch?v=7O84mdxehWY&t=502s) They also tested **ELIZA** (1966)—the program the Count's lodger is named after. ELIZA was picked human **23**/100 and beat one modern model. "She would like that noted." - [8:54](https://www.youtube.com/watch?v=7O84mdxehWY&t=534s) And the line moved: the Turing test "had never been a good test." Conversation, too, had been standing in for something else—Jefferson's "thoughts and emotions felt." - [9:15](https://www.youtube.com/watch?v=7O84mdxehWY&t=555s) Arithmetic, chess, Go, language: four lines, four crossings. Each time the machine did the thing, and people decided the thing had never been thinking after all. ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0420.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0460.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0500.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0540.webp]] ### 9:32 Part Five: Understanding - [9:34](https://www.youtube.com/watch?v=7O84mdxehWY&t=574s) **Part five, understanding.** "Fine… it can do all of that, but it doesn't understand." Best defenders first. - [9:52](https://www.youtube.com/watch?v=7O84mdxehWY&t=592s) **John Searle**, 1980: the **Chinese room**. A man who knows no Chinese follows an English rulebook, slides fluent Chinese answers under the door; outside speakers find him fluent; inside nobody understands. A computer running a program is that man—shuffling symbols by rules never adds up to meaning. - [10:35](https://www.youtube.com/watch?v=7O84mdxehWY&t=635s) **Emily Bender** and **Alexander Koller** (ACL 2020), sharpened as the **octopus**: two friends on islands talk through a seafloor telegraph; a clever octopus learns from message rhythm alone, then impersonates one friend. When a bear appears and she asks how to make a weapon from sticks, the octopus has never seen a stick or a bear—only the shape of the words. That is what people mean by **stochastic parrot** (Bender et al., FAccT 2021). - [11:30](https://www.youtube.com/watch?v=7O84mdxehWY&t=690s) Two charges: Searle's (rules ≠ meaning) and the octopus's (words without the world). Unlike earlier lines, these are built so no amount of *doing* can cross them. - [12:11](https://www.youtube.com/watch?v=7O84mdxehWY&t=731s) So look inside. "Give me a few minutes inside one of these machines. No engineering. I'll keep to the idea." ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0580.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0620.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0660.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0700.webp]] ### 12:20 Interlude: How a Machine Reads - [12:23](https://www.youtube.com/watch?v=7O84mdxehWY&t=743s) A language model is trained to do one thing: given text, **guess the next word**, on more writing than any person could read in many lifetimes. Billions of guesses, billions of nudges. "That does sound like auto complete." - [12:52](https://www.youtube.com/watch?v=7O84mdxehWY&t=772s) Detective-novel test: stop at "And the murderer is…" — a good guess needs a **model of the story**. How do you build one from guessing alone? - [13:13](https://www.youtube.com/watch?v=7O84mdxehWY&t=793s) **Geoffrey Hinton**, pioneer of the networks these machines use, has spent ~40 years on that. Four pieces. - [13:23](https://www.youtube.com/watch?v=7O84mdxehWY&t=803s) **First**: a word becomes a **bundle of features**. Mid-1980s toy model: two family trees, 24 people, English and Italian; only kinship facts. The net invented features for family, generation and branch—nobody told it those. - [14:16](https://www.youtube.com/watch?v=7O84mdxehWY&t=856s) Hinton clip: words as **thousand-dimensional LEGO blocks**—"think of a three-dimensional thing… and say thousand to yourself very loudly." - [14:50](https://www.youtube.com/watch?v=7O84mdxehWY&t=890s) **Second**: a block changes shape with its neighbors. "The **count** counts the candles" — same word, nobleman vs finger-action. Reverse: "She **scrummed** him with the frying pan" — you know roughly what scrummed means from the hole the neighbors leave. - [15:23](https://www.youtube.com/watch?v=7O84mdxehWY&t=923s) **Third**: how blocks find neighbors — **attention** (Vaswani et al., Google, 2017). Every word asks every other word which matter; rounds sharpen meaning. Hinton's picture: little **hands** and **handles** that grip until the passage locks into one structure. - [16:40](https://www.youtube.com/watch?v=7O84mdxehWY&t=1000s) Hinton clip: "That structure is understanding." - [16:59](https://www.youtube.com/watch?v=7O84mdxehWY&t=1019s) **Fourth**: the guess. Old phone autocomplete kept co-occurrence tables; a language model stores knowledge in how blocks bend and shake hands—the next word is the piece that fits. - [17:25](https://www.youtube.com/watch?v=7O84mdxehWY&t=1045s) Hinton conclusion clip: digital agents understand language "in much the same way as people do… They're much more like us than they are like standard computer code." ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0740.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0780.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0820.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0860.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0900.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0940.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t0980.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1020.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1060.webp]] ### 18:01 The Verdict - [18:01](https://www.youtube.com/watch?v=7O84mdxehWY&t=1081s) Back to the two charges. Searle's man had a written rulebook; these machines **grew** theirs from billions of guesses. If the room understands anything, understanding lives in the book—and this book holds a model of the story. Searle's memorize-the-book reply: a man who carried that model in his head "sounds… very much like a man who understands it." - [18:32](https://www.youtube.com/watch?v=7O84mdxehWY&t=1112s) Where **Dreyfus** was right: real expertise is a trained eye from thousands of cases, and no rulebook could capture it. He didn't expect a machine to train its eye the same way. - [19:13](https://www.youtube.com/watch?v=7O84mdxehWY&t=1153s) The octopus is harder, but these machines have read a large share of everything people wrote about bears and sticks; the world leaves its shape in that writing. Checkable: models trained only on text encode **cities roughly as on a map** (Gurnee & Tegmark); an **Othello** model trained only on move lists grew a board picture (Li et al.). Newer models also look at pictures, listen, and act. "The octopus is coming up out of the sea." - [20:03](https://www.youtube.com/watch?v=7O84mdxehWY&t=1203s) **Verdict**: if understanding means building a working model of what the words are about and using it, these machines understand. Hinton is right. How do we know? The same way we know it of people—nobody has opened a skull and found understanding; we watch them follow the plot, catch the joke, guess the murderer. - [20:43](https://www.youtube.com/watch?v=7O84mdxehWY&t=1243s) And the line moves one last time: "Fine. It understands. But it doesn't **really** understand." *Really* asks for the one thing that can't be watched—whether anyone in there feels the penny drop. ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1100.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1140.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1180.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1220.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1260.webp]] ### 21:13 Closing - [21:13](https://www.youtube.com/watch?v=7O84mdxehWY&t=1273s) So does thinking need a brain? For the **doing**—arithmetic, chess, Go, language, and if Hinton is right understanding—none of it needed one. For the **feeling**, nobody has ever had a test: not for a machine, a dog, or the person next to you. - [21:41](https://www.youtube.com/watch?v=7O84mdxehWY&t=1301s) Turing said so in 1950 answering the surgeon: we extend one another a polite convention; why shouldn't a machine get it too? - [21:52](https://www.youtube.com/watch?v=7O84mdxehWY&t=1312s) One mark left on the floor: the oldest. Pascal kept back the will; Jefferson redrew it in 1949. Every wiped line tonight was a stand-in for this one—people drew testable proxies in front of it. "I'll ask once: what makes anyone sure this mark will hold when none of the others did?" - [22:27](https://www.youtube.com/watch?v=7O84mdxehWY&t=1347s) Closing question for comments: "What's the one thing you're certain a machine will never do? Tell me in the comments. I'll keep the list. I've kept such lists before. Good night." ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1300.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1340.webp]] ![[images/AI-Doesnt-REALLY-Understand-Weve-Said-That-Before-t1365.webp]] ## Things mentioned ### People - **Hubert L. Dreyfus** — philosopher; 1965 RAND paper; lost to Mac Hack VI (1967); decades of critique of GOFAI - **Blaise Pascal** — Pascaline (1642, Rouen); Pensées on the arithmetical machine and will - **Herbert A. Simon** (with Allen Newell) — 1957/1958 chess-champion prediction - **Garry Kasparov** — world champion; lost to Deep Blue, May 1997 (and lost a game in 1996) - **Douglas Hofstadter** — revised view after Deep Blue; *Gödel, Escher, Bach* (1979) cited in description - **Piet Hut** — Princeton astrophysicist; 1997 *NYT* Go forecast - **Lee Sedol** — Go champion; AlphaGo match, Seoul, March 2016 (lost 4–1); move 37 - **Alan Turing** — "Computing Machinery and Intelligence" (*Mind*, 1950); Turing test; polite convention - **Geoffrey Jefferson** — Manchester surgeon; Lister Oration, London, 9 June 1949 - **John Searle** — "Minds, Brains, and Programs" (1980); Chinese room - **Emily M. Bender** & **Alexander Koller** — ACL 2020 octopus / form-vs-meaning paper; Bender et al. "stochastic parrots" (FAccT 2021) - **Geoffrey E. Hinton** — distributed representations; IASEAI '25 "What Is Understanding?" clips - **Cameron R. Jones** & **Benjamin K. Bergen** — "Large Language Models Pass the Turing Test" (2025, arXiv:2503.23674) - **Ashish Vaswani et al.** — "Attention Is All You Need" (NeurIPS 2017) - **Wes Gurnee** & **Max Tegmark** — "Language Models Represent Space and Time" (2023) - **Kenneth Li et al.** — Othello world-model paper (ICLR 2023) - **John Tromp** — count of legal Go positions (2016) - **David Silver et al.** — AlphaGo *Nature* paper (2016) ### Machines, programs & games - **Pascaline** — Pascal's brass calculator - **Mac Hack VI** — MIT chess program that beat Dreyfus (1967) - **Deep Blue** — IBM chess machine (~200M positions/s) - **AlphaGo** — DeepMind; policy net, value net, MCTS-style search - **ELIZA** (1966) — Weizenbaum chatbot; namesake of the Count's lodger; scored 23/100 as "human" in the 2025 study - **Go**, **chess**, **Othello** — games used as moving goalposts / probes of world models ### Ideas & phrases - The **line** between machine-doable and mind-only; the word **just** - **Chinese room** (Searle) - **Octopus** / **stochastic parrot** (Bender & Koller; Bender et al.) - **Attention**, feature bundles, hands-and-handles metaphor (Hinton) - The untestable residue: **"really"** / felt experience / will ### Links (from the video description) - Hinton, "What Is Understanding?", IASEAI '25: https://www.youtube.com/watch?v=6fvXWG9Auyg - Dreyfus, "Alchemy and Artificial Intelligence" (RAND P-3244): https://www.rand.org/pubs/papers/P3244.html - Jones & Bergen Turing-test paper: https://arxiv.org/abs/2503.23674 - Bender & Koller ACL 2020: https://aclanthology.org/2020.acl-main.463/ - Stochastic Parrots (FAccT 2021): https://doi.org/10.1145/3442188.3445922 - Attention Is All You Need: https://arxiv.org/abs/1706.03762 - Gurnee & Tegmark: https://arxiv.org/abs/2310.02207 - Li et al. Othello: https://arxiv.org/abs/2210.13382 - Prior episode on the channel: *The Math OpenAI Didn't Publish* — https://youtu.be/BX9cqMmhoU0 ## Biographies ### Count Grey Matter (host / channel) YouTube channel **Count Grey Matter** ([@CountGreyMatter](https://www.youtube.com/@CountGreyMatter), channel ID `UCi7xCjT4T7BdJ640d4o3gcA`). At the time of this video it had on the order of **~5.5k** subscribers. The host appears as a pixel-art gothic nobleman ("the Count") in a study, with a lodger named after **ELIZA**. The channel's videos are essay-style pieces on AI, philosophy and related history; the previous episode cited in the description is *The Math OpenAI Didn't Publish*. No real-name credit is given in the description for this episode. ### Hubert L. Dreyfus (1929–2017) American philosopher known for Heidegger-influenced critiques of artificial intelligence. His RAND paper *Alchemy and Artificial Intelligence* (P-3244, 1965) argued early symbolic AI overpromised; he later lost a chess game to **Mac Hack VI** at MIT (1967) and continued arguing that expertise is situated and not reducible to explicit rules (*Mind over Machine*, with Stuart Dreyfus, and later essays such as "Making a Mind Versus Modeling the Brain," *Daedalus* 1988). ### Geoffrey E. Hinton British-Canadian cognitive psychologist and computer scientist, a central figure in deep learning and neural networks (sometimes called one of the "godfathers" of AI). Early work on distributed representations includes the 1980s family-tree experiments described in the video. He left Google in 2023 to speak more freely about AI risks; the clips here are from his IASEAI '25 talk "What Is Understanding?" ### John Searle American philosopher of mind and language. His 1980 paper "Minds, Brains, and Programs" introduced the Chinese room argument against the claim that running the right program is sufficient for understanding or mindedness. ### Emily M. Bender Computational linguist (University of Washington). Co-author of the ACL 2020 paper with Alexander Koller on form vs. meaning, and of the 2021 FAccT paper that popularized the phrase **stochastic parrots**. ### Alan Turing (1912–1954) Mathematician and founding figure of computer science. "Computing Machinery and Intelligence" (*Mind*, 1950) proposed the imitation game (Turing test) and answered contemporary objections, including Jefferson's, with the "polite convention" that others think.