The Transformer: Attention Without Recurrence
The Transformer replaced recurrence with self-attention, enabling massively parallel training — the architecture behind GPT, BERT and modern LLMs.
Explainer · 1:40 · 4 chapters
Why a language model feels like it understands, what it is actually doing, and three questions that replace “does it understand?”.
Why do language models feel like they understand? This explainer gives the mechanism in under two minutes. Your brain treats text that is easy to read as true — processing fluency — and a language model produces the most fluent text ever written. It does not look facts up. It ranks the most plausible next token, using patterns learned from human text. Training compresses trillions of words into a few hundred billion numbers, which keeps patterns and discards the rare details where truth often lives, and a model has no window into what it threw away.
The second half is about us. Words like wants, thinks, understands and lies import a human, because brains evolved to read minds and find intention even in a matrix of numbers. That leaves two opposite errors: treating a statistical engine as an oracle, or dismissing it as “just autocomplete” and missing what it can do. The film's alternative to asking “does it understand?” is three working questions.
You think you understand AI. Almost nobody does, and the reason isn't intelligence. It's your brain. Your brain runs a shortcut: easy to read feels true. That's processing fluency, and a language model produces the most fluent text ever written. So your truth detector fires before you check a single fact. A language model doesn't look up facts.
It ranks the next token, the most plausible continuation built from patterns in human text. To do that, it compresses: trillions of words squeezed into a few hundred billion numbers. Compression keeps patterns and discards what's rare, and truth usually lives in the rare parts. It can't tell you what it threw away. No model has a window into its own ignorance, so when it doesn't know, it doesn't hesitate. It hands you the most plausible sentence available: confident, fluent, wrong.
Now, the deeper problem. We say the model wants, thinks, understands, lies. Each of those words smuggles in a human. Your brain evolved to read minds. It finds intention in a shadow, in a face on Mars, in a matrix of numbers. So we get two ways to be wrong: treating a statistical engine as an oracle, or calling it “just autocomplete” and dismissing what it can actually do. Same mistake: explaining a new thing with an old word. Drop the question, “Does it understand?”
Ask instead: What does it do reliably? Where does it fail? And how would I know? Read it like an overconfident colleague who never says “I don't know.” Verify everything. Keep your name on the work. That's the real skill now: not using AI, but knowing exactly what you're holding. Subscribe, and next time we go deeper.
Transcribed with Whisper (small.en) and corrected by hand against the narration and the on-screen text. Click any sentence to play from there.
The Transformer replaced recurrence with self-attention, enabling massively parallel training — the architecture behind GPT, BERT and modern LLMs.
An AI-generated review of GPT-3, the 175-billion-parameter language model that showed scale alone can unlock in-context learning without any gradient updates.
An AI-generated review of retrieval-augmented generation (RAG), a framework that grounds language models in external knowledge to improve factual accuracy.
An AI-generated review of InstructGPT, the paper that introduced instruction fine-tuning with human feedback (RLHF) and defined how aligned LLMs are built.