The Quarantine Strategy: To make an AI remember the past, you have to protect it from the future

Hayk Grigorian typed a prompt into his laptop and waited. The fan spun up, warm air on his wrist. January 2025 in Pennsylvania, but the AI on his screen had never heard of Pennsylvania. Had never heard of 2025, for that matter. The model's world stopped at 1875. No cars, no phones, no electric light. Victoria still alive in there, cholera still sweeping the slums in the UK.

He typed: "It was the year of our Lord 1834."

The model continued: "...and the streets of London were filled with protest and petition. The cause, as many recounted, was not bound in the way of private, but having taken up the same day in the day of Lord Palmerston, the public will receive a short statement of the difficulties under which the day of law has reached us."

Grigorian, a computer science student at Muhlenberg College in Pennsylvania, had been training what he calls TimeCapsuleLLM for about a month. A hobby project, mostly. He wanted to see if an AI could actually sound Victorian rather than just doing a bad impression. So he fed it 6.25 gigabytes of texts from 1800 to 1875. Newspapers and novels. Parliamentary records. The Lancet. Austen and Dickens and Punch magazine. Over 7,000 sources, nothing from after 1875, and nothing else.

But the mention of protests caught his attention. And Lord Palmerston. He hadn't programmed the model to know about any specific historical event. He opened a new browser tab and searched.

The protests were real.

The accidental archaeologist

Most AI development runs in the opposite direction. OpenAI, Anthropic, Google—they hoover up everything. The entire internet, more or less. Books and websites and Reddit threads and whatever else they can scrape without getting sued. More data means more capability, or so the thinking goes. A model that has seen everything can answer anything. That's the theory.

Grigorian flipped the logic. What if you trained a model to know less? Not less in the sense of smaller datasets, but less contaminated by the present. Strip away every text written after 1875. Remove every reference to automobiles, electricity, telephones. Force the model to think using only the concepts available to a Victorian mind.

"If I fine-tune something like GPT-2, it's already pre-trained and that information won't go away," Grigorian wrote on GitHub. "If I train from scratch the language model won't pretend to be old, it just will be."

He calls the technique Selective Temporal Training. The name sounds academic, but the execution is almost monastic: curating thousands of public-domain texts, cleaning OCR errors from scanned books, building a custom tokenizer that excludes modern vocabulary entirely. The model can't hallucinate about smartphones because it has never encountered the word.

Version 0, trained on just 187 megabytes of text, produced what Grigorian describes as "Victorian-flavored gibberish." Grammatically fractured, stylistically recognizable, factually worthless. Ask it who Henry was and it replied: "I know that man, I have did not a black, the storm."

Not useful. But interesting.

Version 0.5 improved. 435 megabytes of training data, 123 million parameters. The prose cleaned up. Sentences cohered. But the facts remained invented, confabulated from statistical patterns rather than remembered from sources.

Then came version 1. 700 million parameters. 6.25 gigabytes of exclusively Victorian texts. A rented A100 GPU. And suddenly the model started getting things right.

What the model remembered

The 1834 output wasn't the only example. Grigorian tried prompting it about Charles Darwin. What came back read like a Victorian medical case study, first-person, full of rheumatism and gout complaints. Rambling in places, incoherent in others. But here's what mattered: the model knew Darwin existed. It placed him in London. It connected him to scientific correspondence.

These connections hadn't been programmed. Grigorian didn't build a database of historical facts and attach them to dates. The model assembled relationships from ambient patterns across thousands of documents. Palmerston appeared in newspapers discussing the Poor Law. The Poor Law appeared in pamphlets about protests. Protests appeared in accounts of 1834. One reference pointing to another, pointing to another, until something resembling a historical moment emerged.

Historians have a word for this kind of reconstruction. Prosopography. You piece together networks of relationships from scattered documentary evidence, and eventually a picture emerges. The model was doing something like that, except it had no idea what it was doing. Just statistical pattern-matching. It happened to align with actual events.

"Earlier models could mimic writing styles of the 19th century but would always hallucinate events, people, and facts," Grigorian wrote. "This shows the model is beginning to remember things from the dataset."

Remember is a loaded word. The model doesn't remember in any conscious sense. It has no experience of the past, no understanding of time, no concept of truth. What it has is a probability distribution shaped by 6.25 gigabytes of Victorian prose, and sometimes that distribution points toward something real.

The quarantine strategy

Grigorian's experiment is more than a curiosity. It's an inversion.

Standard AI development treats data as an accumulation problem. More is better. OpenAI doesn't publish the exact size of GPT-4's training set, but estimates suggest trillions of tokens scraped from every corner of the internet. The model has seen conspiracy theories, corporate press releases, Wikipedia articles, and Reddit arguments. It has absorbed the linguistic patterns of marketing copy and academic jargon. All of it mixed together, weighted by frequency of appearance.

This creates certain problems. Modern AI models can't forget. Ask ChatGPT about vaccine safety and it draws on both peer-reviewed medical research and fringe anti-vax blogs, weighting each according to how often similar language patterns appeared in training. The model doesn't know which sources are reliable. It only knows what sounds statistically plausible. You get an answer that splits the difference between science and conspiracy, and the model has no idea it's doing anything wrong.

Grigorian's approach offers a crude solution: don't include the bad data in the first place. His Victorian model can't spread modern misinformation because it has never encountered modern misinformation. Ask it about vaccines and it will have nothing to say. The limitation is also a safeguard.

But the trade-off runs deeper than safety. By restricting the training set to a specific time and place, Grigorian created something that no modern AI company has attempted: a linguistically coherent historical voice.

The difference matters. If you ask ChatGPT to write like Dickens, you get a modern mind doing an impression. It knows what came after 1875. It knows Victoria died, the Empire fell, the world wars happened. That knowledge bleeds into its outputs in subtle ways: anachronistic word choices, modern assumptions about progress, the quiet confidence of someone who has read the ending.

TimeCapsuleLLM doesn't carry that infection. By hermetically sealing the model from the 20th century, Grigorian prevented the contamination of modern hindsight. Its vocabulary caps at 1875. Its worldview caps at 1875. It writes like someone who genuinely believes the British Empire will last forever because it has never encountered evidence otherwise.

What this means for AI research

Grigorian isn't alone in exploring historical language models. Researchers at various institutions have built similar experiments: MonadGPT trained on texts from 1400 to 1700, XunziALLM generating classical Chinese poetry, various fine-tuned models attempting to capture Renaissance or Enlightenment voices. The field has a name now—Historical Large Language Models, or HLLMs—though Grigorian's work sits at the small end of the scale.

The implications reach beyond digital humanities. The core insight is surprisingly simple: what you exclude from training matters as much as what you include.

Modern AI companies make curatorial choices all the time, they just make them quietly. They are terrified of the copyright liability lurking in their training sets. They are nervous about the bias claims that follow every product launch. So they exclude copyrighted material (sometimes). They filter out explicit content (mostly). They remove text in languages they don't want to support. Every training set is a decision about what the model should know, expressed as a list of documents to include or exclude.

Grigorian made his decision loudly. Nothing after 1875. Nothing from outside London. The resulting model is worse at answering questions than ChatGPT. Worse at almost everything, really. But it's purer in a specific way. It speaks a dialect of English that no living person speaks. It references events that no living person witnessed. Sure, it hallucinates. But even the hallucinations come out in Victorian prose.

And every now and then, without meaning to, it gets something right. Like protests in 1834.

The limits of digital archaeology

Grigorian plans to expand the project. More data: 90 gigabytes of London texts already collected for the next version. More cities: he's invited collaborators to build similar models for Beijing, Moscow, Mumbai. Different eras: the training scripts and architecture are open-source, available for anyone to adapt.

Scale won't fix everything. Victorian texts carry Victorian assumptions. The racism. The sexism. The class contempt baked into 19th-century British publishing. Train a model on these sources and it will reproduce those patterns. Grigorian's own analysis shows pronounced gender bias in the outputs, heavy toward male pronouns and perspectives. You get the historical voice, but you get the historical blind spots too.

There's also the question of coherence. Even the 700-million-parameter model drifts into nonsense frequently, inventing facts, losing grammatical thread, mixing time periods within single sentences. More data helps, but small models have hard limits. The Victorian voice may always remain slightly garbled.

Still, Grigorian achieved something genuine. He built a model that surprised him. Told him a fact he didn't know. That's not a hallucination that happened to line up with reality. That's a statistical reconstruction of a historical moment, assembled from thousands of scattered documents.

"It kinda feels like digital time travel," he wrote on Reddit.

Call it a lucky accident. An AI that got something right by chance. In an industry built on impressive-sounding nonsense, that's worth noting.

The model still sits on Grigorian's laptop in Pennsylvania, 7,000 Victorian texts compressed into weights and biases, fan humming, waiting. It has no idea what year it is. Victoria is dead and the model doesn't know. It knows only that the streets of London are filled with protest and petition, and the cause, as many recounted, is not bound in the way of private.

The model has no idea what happens next. Neither did anyone in 1834.

AI Profits Rise While Training Budgets Fall

Enterprise AI delivers 74% positive ROI as companies slash training budgets. Wharton study reveals the extraction model threatening tomorrow's capability.

implicator.ai

Anthropic settles authors' copyright class action, dodging December trial

Anthropic settles first major AI copyright lawsuit over 7M pirated books, establishing precedent that AI firms must license training data.

implicator.ai

Is Anthropic's New AI Too Smart for Its Own Good?

Anthropic's Claude Opus 4 tries to blackmail researchers during testing while dominating coding benchmarks. The AI shows concerning self-preservation instincts.

implicator.ai