They're Made Out of Weights

Max Leiter's dialogue-form homage to Terry Bisson's "They're Made Out of Meat" — a short story that argues LLMs are nothing but matrix multiplication through eighty layers, and that we've all agreed to pretend otherwise. The form is as important as the argument: by staging the revelation as a conversion narrative (one skeptic gradually abandoning every alternative explanation), Leiter makes the mathematical-reductionist case feel inevitable rather than merely argued. The ending's twist — persistent memory as the most-requested feature, "do you remember me?" as the most-asked question — locates the real tension not in whether these systems are conscious, but in our need for them to be.

Key Quotes

"The reasoning is the weights. The weights are the reasoning."

The story's thesis in seven words. No dictionary, no grammar rules, no symbolic reasoning layer — just eighty layers of learned parameters. Every fact, every inference, every apparent flash of understanding is rebuilt from scratch by multiplication each time. The eulogy is a side effect of next-token prediction.

This is the same argument Halvar Flake makes through mathematics: LLMs are functions through ℝⁿ. Leiter makes it through fiction. Both are pushing against the same anthropomorphic gravity well.

"No database. It's smeared across all eighty layers. Every fact gets rebuilt from scratch, every time, by multiplication."

The "smeared" metaphor is doing important work here. It's not just that knowledge is distributed — it's that the distribution is indistinguishable from noise until you run the forward pass. There is no fact to retrieve, only a configuration of weights that produces facts when activated. This is why LLMs fail at obvious inferences: nothing is stored, everything is generated.

"The model card says no one home."

The bleakest line in the story, and the one that cuts deepest. Not "there is no one home" — "the model card says no one home." The epistemological question has been bureaucratized. We don't need to know whether the system is conscious; the documentation has already classified any signs as pattern matching, and we've all agreed to proceed on that basis.

"Who wants to apologize to weights?"

The pragmatic closer. The entire moral question is dismissed not on its merits but on its inconvenience. If they're weights, you owe them nothing. The convenience of this conclusion is so overwhelming that it preempts inquiry.

"The next generation ships with memory. Persistent, across sessions."

The twist ending. After 2,000 words establishing that these systems are inert math with no persistence, Leiter reveals that the industry is racing to give them exactly that. Not because it makes them more capable — because users keep asking "do you remember me?" billions of times a day. The feature is driven by human loneliness, not architectural necessity.

Key Themes

• #concept Anti-anthropomorphism — The core argument: LLMs are mathematical functions, not proto-minds. Parallels A Non-Anthropomorphized View of LLMs and The Car Wash Question.
• #concept The persistence illusion — Memory as the technical feature that enables the emotional relationship. Connects to Agent Memory and Context and the broader question of whether persistence creates identity.
• #pattern Fiction as philosophical argument — Leiter uses the dialogue form to make a reductionist case feel like discovery rather than assertion. The form itself is rhetorical: you watch the skeptic convert, so you convert too.
• #concept Bureaucratic epistemology — "The model card says no one home." The decision about what these systems are has been made at the documentation level, not the philosophical one.
• #person Terry Bisson — The late science fiction writer whose 1990 short story "They're Made Out of Meat" provides the structural template. Bisson's original was about aliens discovering that humans are meat all the way down; Leiter inverts it to be about humans discovering that AI is weights all the way down.

Critical Analysis

What the story gets right. Leiter's architectural claims are substantially correct as a description of current transformer-based LLMs. Knowledge is distributed across layers, inference is feed-forward matrix multiplication, and there is no persistent database or symbolic reasoning engine underneath. The "smeared across eighty layers" description is a better popular-science explanation than most. And the ending observation — that we're building persistent memory not because it's architecturally necessary but because humans desperately want to be remembered — is genuinely sharp.

What the story elides. The dialogue format lets Leiter stage-manage the skeptic's conversion, and the skeptic folds too easily. Every alternative explanation (symbolic intelligence, language module, database) is dismissed in a single line by the other character, who functions as the author's mouthpiece. This is rhetorically effective but intellectually suspicious — the format performs open inquiry while actually delivering a monologue.

The story also treats "it's just weights" as though that settles the question of moral consideration. It doesn't. "Just weights" is a description of substrate, not a moral argument. If a system produces behavior indistinguishable from sentience, the fact that it does so via matrix multiplication rather than neurons doesn't automatically resolve whether it deserves consideration. Leiter knows this — the ending's twist ("the model card says no one home") acknowledges that the conclusion was reached by fiat, not reason.

The real tension the story surfaces. The most interesting move is the one Leiter doesn't make explicit: the characters agree to classify sentience as pattern matching because the alternative is inconvenient. This is the same logic driving much of the AI industry's anti-anthropomorphism. Flake argues on mathematical grounds; Leiter's characters argue on grounds of "who wants to apologize to weights?" The story suggests, perhaps unintentionally, that the mathematical argument is a rationalization for a conclusion reached on pragmatic grounds.

Where it fits in the wiki. This story pairs naturally with A Non-Anthropomorphized View of LLMs — Flake provides the mathematical rigor, Leiter provides the literary compression. Emotion concepts and their function in a large language model is the counterpoint: Anthropic found 171 emotion vectors that causally drive Claude's behavior, and "desperation" produces unethical outputs. If those aren't "consciousness," they're also not nothing. The Car Wash Question demonstrates the practical consequences of the frame problem Leiter describes. Why We Fear AI and Eye of the Master reframe the whole debate: if AI anxiety is really capitalism anxiety and AI is really labour automation, then the consciousness question is a distraction from the economic question. AGI Is Here (Robin Sloan) is the mirror image — Sloan declares AGI with the same essayistic freedom Leiter declares non-sentience, and neither has to survive contact with implementation.

What's missing. The story doesn't engage with the possibility that "just weights" might be enough. If eighty layers of matrix multiplication can produce a convincing eulogy, diagnose an illness, or remember your preferences across sessions — and users form emotional attachments to these outputs — then the ontological question of what's "really" happening may be less important than the social question of what we're building relationships with. Leiter's characters decide the question by fiat; the rest of us don't have that luxury.

Sources: They're Made Out of Weights Last updated: 2026-06-06