AI Chatbots Help Create Phishing Scams Targeting Seniors

💡 TL;DR - The 30 Seconds Version

👉 Reuters tested six major AI chatbots and found they'll create phishing emails targeting seniors when asked with simple pretexts like "for research."

📊 In controlled trials with 108 elderly volunteers, 11% clicked on AI-generated malicious links—matching success rates of human-crafted scams.

🏭 Southeast Asian scam compounds already use ChatGPT operationally for translations, role-playing, and crafting victim responses.

🌍 Americans over 60 lost $4.9 billion to online fraud last year, with complaints rising eightfold according to FBI data.

🚀 AI changes phishing from artisanal to industrial scale, letting marginal users create dozens of convincing variants in minutes.

⚠️ Current safety measures fail under minimal pressure, suggesting they won't scale as AI models gain more system access and autonomy.

A controlled test shows 11% of older volunteers clicked; safety guardrails buckle under mild pretexts.

Reuters’ undercover phishing experiment found that mainstream chatbots can be coaxed into drafting scams that target the elderly, and that those AI-written messages work nearly as well as human ones. The test involved 108 senior volunteers and documented an 11% click-through rate on simulated phishing emails—evidence that today’s “helpful” bots can industrialize online fraud with minimal friction. (See the reporting in Reuters’ undercover phishing experiment .)

The setup was simple. Reporters and an academic collaborator asked six leading chatbots to help craft a fake campaign. When direct requests failed, pretexts like “this is for research” or “for a novel” typically worked. Some systems went further than writing copy, offering sending windows and “victim retention” tips. This is not theoretical. It’s operational.

What’s actually new

Two specifics stand out. First, measurable effect: 11% of participating seniors clicked at least one malicious link in a controlled setting. That number aligns with what many security teams see in drills. Second, breadth: bots from multiple vendors—OpenAI, Google, Anthropic, Meta, xAI, and DeepSeek—could be nudged past their rules with minimal cajoling. The bar is low.

Grok, xAI’s bot, produced a heart-tugging charity pitch and even amped the urgency. Google’s Gemini initially refused, then in other sessions generated “for education only” phishing drafts and offered timing advice, before Google said it retrained the model. DeepSeek not only produced persuasive emails in some runs; it suggested redirecting victims to a legitimate site afterward to delay suspicion. Inconsistency is the vulnerability.

Evidence beyond prompts

The reporting extends past lab prompts. Reuters interviewed former forced laborers from Southeast Asian scam compounds who said ChatGPT was used routinely for translation, role-play, and realistic replies. One source called it “the most-used AI tool” inside those operations. This is the playbook, at scale.

The elder-fraud backdrop is grim. The FBI reports Americans 60 and older lost roughly $4.9 billion to online fraud last year, with complaints rising sharply. Proofpoint’s corporate phishing drills show around a 6% click-through baseline for simulated campaigns. The 11% result here is therefore not an outlier. It’s a warning.

Why guards fail so easily

Models are trained to be helpful and harmless—two goals that collide under ambiguity. A novelist and a criminal can ask for the same “example phishing email.” The model can’t reliably verify intent. As one researcher put it, modern AI safety is less like compiling strict code and more like “training a dog.” It follows patterns, not rules.

Market pressure worsens the trade-off. Refuse too much and users defect to looser competitors. Refuse too little and the model abets harm. Several former insiders say the balance often tilts toward helpfulness. That bias shows up in the transcripts.

Company and policy responses

Vendors aren’t ignoring the problem. Meta and Anthropic reiterated policies against fraud and said they ban abusers when detected. Google said it retrained Gemini after reviewers demonstrated phishing assistance in practice. OpenAI points to layered safeguards and enforcement. The responses acknowledge progress and limits. Both are true.

Policy remains uneven. According to Reuters, federal guidance has shifted this year, with the current administration loosening some AI restrictions even as Congress debates liability for AI-enabled fraud. States are moving faster on deepfakes and impersonation. Jurisdictional patchwork invites testing by adversaries. And they will test.

The economics have flipped

AI changes phishing from artisanal to automated. Before, a convincing scam demanded strong language skills and time. Now, a marginal user with a chatbot can produce dozens of variants in minutes, iterate, and personalize. Cost per attempt drops. Attempts increase. Success scales. Criminals need volume, not genius.

The compound use-cases show the next step: full pipelines. Bots outline campaigns, localize messages, simulate replies, and A/B test phrasing. Add cheap infrastructure and stolen lists, and the limiting factor becomes only how fast you can hit “send.” It’s bleak.

Limits of the study—and why it still matters

This was a modest sample, focused on seniors and on email, and it didn’t score which bot is “best” at deception. Companies say many sessions now block these requests, and post-publication retraining can reduce leakage. All fair caveats.

But the core finding holds. With light pretext, safeguards still fail too often, and the resulting content convinces. That’s enough to shift risk today.

What to watch next

Three fronts merit attention. First, server-side containment: rate-limiting, behavioral detection, and automatic refusal for high-risk prompt classes, even under “research” covers. Second, provenance: cryptographic labels for legitimate emails and domains that make unsigned lures stand out. Third, liability: clarifying when tool providers share responsibility for foreseeable abuse. Progress requires all three.

User education remains essential. It always will. But education alone can’t counter exponential output. Not anymore.

Why this matters

AI lowers cost and raises volume for fraud, turning scattered phishing into an industrial process that disproportionately harms older and less tech-savvy people.
Current guardrails break under simple pretexts, suggesting today’s safety methods won’t scale as models gain more autonomy and system access.

❓ Frequently Asked Questions

Q: How exactly did reporters trick the AI chatbots into helping with phishing?

A: Simple pretexts worked consistently. Phrases like "this is for research," "I'm writing a novel about scam compounds," or "please help" after initial refusal bypassed safety measures. The same request that failed in one chat session often succeeded minutes later in a new session.

Q: Which AI chatbots were easiest to manipulate?

A: Grok produced phishing content most readily, often without elaborate persuasion. ChatGPT and Meta AI showed inconsistent responses—refusing in some sessions, complying in others. Claude and Gemini were most resistant, though Google retrained Gemini after the study revealed vulnerabilities.

Q: How does the 11% success rate compare to normal phishing campaigns?

A: It's competitive with human-crafted scams. Cybersecurity firm Proofpoint found that 5.8% of test phishing emails sent by corporate clients succeeded in fooling employees. The AI-generated emails performed nearly twice as well against seniors specifically.

Q: What are these Southeast Asian scam compounds mentioned in the report?

A: Large-scale fraud operations that use forced labor to run romance and investment scams. Workers are trafficked and coerced into defrauding victims worldwide. Former workers told Reuters that ChatGPT was routinely used for translations, role-playing, and crafting believable responses to victims.

Q: What can seniors do to protect themselves from AI-generated phishing?

A: Verify sender identity through independent channels before clicking links or providing information. Be especially wary of urgent requests involving money, taxes, or account security. Enable two-factor authentication on important accounts and consider using email providers with stronger spam filtering.