This is Brad DeLong's Grasping Reality—my attempt to make myself, and all of you out there in SubStackLand, smarter by writing where I have Value Above Replacement and shutting up where I do not… Please: Enough with the Claims That Modern Advanced Machine Learning Models Hallucinate Only Rarely"Rise of the Sparrows", disappearance of the world model: what MAMLM hallucinations are trying to tell us. Treating large language models as semi-sober research assistants misunderstands what they...Rise of the Sparrows, disappearance of the world model: what MAMLM hallucinations are trying to tell us. Treating large language models as semi-sober research assistants misunderstands what they are: compressed correlation engines, not minds. A modern chatbot confidently listed the “second volume” of a fantasy series as a book that doesn’t exist, producing a botched fantasy bibliography plus an unhinged plot summary. A week later, it produced a a lovingly described but substantially imaginary state of Europe during World War II. This is what you deserve to get when you ask a correlation machine to impersonate a thinking entity with a world model substantially containing an accurate Visualization of the Cosmic All. You don’t just risk getting a detail wrong—you risk wandering into a parallel universe of invented novels and impossible borders; leading one to ask what, exactly, these systems are doing under the hood; and getting nothing in response but that ASI is coming any moment now.It requires very careful iterative prompting to get MAMLMs into conformity with reality—and often even that does not work well enough. One example: We don’t even have to get into the strange political boundaries of axis allies. (Or just what is that country sandwiched between Poland and Romania supposed to be? And is that an Adriatic Republic separate from Italy spanning Venetia and Dalmatia? And what is that map stretching/projection anyway? And how has Turkey managed to conquer all the land up to the crest of the Caucasus Mountains?) You look first at the absence of Vichy France (then in a situation parallel to then-Spain), then at the Nazi occupation of Sweden, third at the failure to register Finland as an active Nazi-allied belligerent, fourth at the independent allied power of southern Norway, and fifth at the Nazi occupation of Great Britain north of historic Wessex. Or there was the time last week when I asked it for the title of the second volume of Elizabeth Wheatley’s <https://www.elisabethwheatley.com/> “Daindreth” fantasy series:
Ummm… No. Simply not. The books are:
There is a related novel, Bastart’s Honor, and a novella, “The Archduke”. But there is no rise, no sparrows, no outrider, and no champion. And the plot/theme summary is invented bullshit as well.
None of these help me understand where Rise of the Sparrows comes from. Or where the words “Outrider” and “Champion” in book titles come from. “Daindreth” is an odd word. My guess is that there are less than 6000 webpages in which the words “Daindreth” and “Elisabeth Wheatley” both appear. A niche-but-not-tiny fantasy romance series typically generates: (i) one canonical page per retailer per edition/format, (ii) one canonical page per book database entry, and (iii) a long tail of reviews, lists, reading trackers, scraped metadata mirrors, and forum/reddit chatter. Who on the internet was ever having a conversation about Elisabeth Wheatley’s “Daindreth” series that then jumped to talking about Sarina Langer—whose first volume of Ar’Zac is called Rise of the Sparrows—for the MAMLM ChatBot to glom onto? Lack of a world model and attempts to substitute for that lack with correlation matrices induces strange behavior indeed. And I am not even going to try to imagine what in the training data could possibly have given rise to the map. I mean, I understand how in drawing hands you can get enthuiastic about fingers and keep drawing more of them. But this flummoxes me. When a chatbot confidently tells you that Rise of the Sparrows is the first volume of a series it has just fabricated, or produces a map in which Turkey owns the Caucasus but Nazi panzertruppen drink from the Bosporus, and southern Norway fights on in company with historic Wessex alone, you are not seeing a tiny edge case. You are seeing the core logic of a system that has patterns instead of facts and correlations instead of a model of reality. This is how MAMLMs actually operate, why “compression” is a treacherous metaphor, and how RLHF and prompt engineering polish—but do not cure—the underlying tendency to make things up. The conclusion is uncomfortable but important: without a world model, correlation matrices will always hallucinate—and often in ways we can’t predict and can’t prune out, unless we already know what the answers are that we are purporting to be trying to get. If reading this gets you Value Above Replacement, then become a free subscriber to this newsletter. And forward it! And if your VAR from this newsletter is in the three digits or more each year, please become a paid subscriber! I am trying to make you readers—and myself—smarter. Please tell me if I succeed, or how I fail…##please-enough-with-the-claims-that-modern-advanced-machine-learning-models-hallucinate-only-rarely |
Please: Enough with the Claims That Modern Advanced Machine Learning Models Hallucinate Only Rarely
Monday, 16 February 2026
Subscribe to:
Post Comments (Atom)



No comments:
Post a Comment