The computer-phobic AI prophet whose warnings came true

A 1998 insight explains why today’s models generate so well—and judge so poorly.

Margaret Boden died on July 18 at 88, a pioneer of cognitive science who liked to say she “couldn’t cope with the damn things.” She grumbled about her Mac, yet she saw with unusual clarity what computers could and couldn’t do. That tension defines her legacy, captured crisply in a line from 1998 that now reads like a field note from the future, as summarized in the Nature obituary on her creativity.

What she actually foresaw

In a paper for Artificial Intelligence (1998), Boden drew a bright line between making ideas and judging them. She argued that AI would find it far easier to generate new combinations than to evaluate their worth. That claim has aged into a diagnosis. It maps directly onto the habits of modern large language models, which spin fluent text on command yet often miss the mark on accuracy or relevance.

Her point was structural, not scolding. Creation and curation are different jobs. The second is the grind.

Three kinds of creativity, one missing brake

Boden organized creativity into three modes. First, combinational: mix familiar elements and you get something that feels fresh—the internet’s daily trick. Second, exploratory: search inside a rule-bound space, like a jazz solo that stays within the changes or a chemist extending a known family. She cited software such as Impro-Visor to show how far structured exploration can go.

The third mode, transformational, is rarer. It rewrites the rules themselves. That leap is still mostly human. One line captures the asymmetry: making is easy; meaning is not. Keep that in mind.

Where the gap turns into trouble

The latest models churn out copy at industrial speed. They remix ideas, riff across domains, and never tire. The weak spot shows up when the answer has to be right for a particular context. Courts have received citations to cases that don’t exist. Students get polished errors. Newsrooms meet plausible fakes that read like house style.

The shortfall isn’t a matter of how many words the model can produce. It’s about knowing which words should survive contact with reality. Whether an answer is acceptable depends on the questioner, the risk, and the standards in play. Pattern matching helps; it doesn’t decide what’s true. People still do.

Boden anticipated that stubborn gap—and why bigger models or more data wouldn’t simply erase it. Evaluation draws on context and values. Those are learned, argued over, and situational. They aren’t just statistics.

The builder as much as the theorist

Boden’s influence wasn’t confined to publications. At the University of Sussex she fused psychology, philosophy, linguistics, and computing into a common home when those fields barely spoke. She co-founded what became COGS, served as its first dean, and kept teaching late into life. The through-line was institution-building for ideas that didn’t fit tidy silos.

Her career brought formal recognition as well. She was appointed OBE in 2002, served as the British Academy’s vice-president from 1989 to 1991, and received the ACM/AAAI Allen Newell Award in 2017. Colleagues often note the honors second and the mentorship first.

Cool eye on companionship and hype

Boden warned against mistaking fluent text for felt experience. She worried that “care-bots” could soothe lonely people while eroding dignity if we pretend the comfort is mutual. The sentences would be smooth; the sentiment would be empty. That wasn’t anti-tech hand-wringing. It was an ethical boundary drawn in plain language.

She also kept her distance from sentience talk. Programs can pursue goals without wanting anything. That distinction drains sci-fi heat and clarifies real risks: capability without intention behaves differently than ambition. The policies should follow that reality.

A frame that still guides product and policy

Read Boden as a handbook for deployment. If evaluation is the bottleneck, you don’t drop raw generators into high-stakes settings and hope scale sorts it out. You add guardrails, expertise, and feedback loops. You measure precision and consequence, not just throughput. You prove reliability in one domain before jumping to the next. Then you keep proving it.

Her distance from day-to-day coding helped more than it hurt. By studying minds before machines, she picked which bets would compound and which blind spots would spread. That’s why her work has outlasted several waves of AI fashion. It is practical philosophy.

Why this matters:

• The structural gap Boden named—generation racing ahead of evaluation—now drives failures in law, education, media, and safety-critical software.
• Treating evaluation as first-class shifts AI strategy toward human oversight, domain constraints, and accountability metrics over raw fluency.