IMPLICATOR
Jacob Coxon sat at a table before New York City officials at City Hall on October 5. The former Anthropic researcher had left the company last month over concerns about the systems it was building. “We do not know how to control any AI system yet,” Coxon said.
Lawmakers were considering a kill switch.
The October 5 hearing examined a proposed city requirement that AI models undergo outside validation and include a human-operated shutdown capability.
What Changed
- New York City lawmakers examined ten AI proposals on October 5; eight were still awaiting formal introduction.
- Menin’s lead bill would require outside validation and a human-operated model shutdown capability, with a $25,000 per-instance civil penalty.
- Coxon urged slower frontier-model development; the 2026 international safety report found no current loss-of-control capability.
- BetaNYC supports validation but says open-weight models and a whistleblower reward need clearer rules.
AI-generated summary, reviewed by an editor. More on our AI guidelines.
What the shutdown requirement covers
Council Speaker Julie Menin unveiled ten legislative proposals on September 25. As of October 5, eight of the ten items had not yet been formally introduced and were set for introduction October 8. The hearing put them under scrutiny. It did not enact them.
Her validation bill would prohibit marketing, selling or deploying an AI model in New York City without an independent assessment. Validators would examine accuracy and bias, alongside privacy and safety risks, and disclose financial interests in the models they assess.
The same bill defines shutdown capability as allowing a human operator to cause a model to stop functioning temporarily or permanently. An outside validator would have to verify that capability. Menin’s lead validation proposal carries a $25,000 civil penalty for each instance of missing or falsified validation, with liability for a noncompliant business and a validator who falsifies validation. That bill would take effect 180 days after enactment.
Coxon described kill switches as potentially useful in the short term but unlikely to ensure long-term safety, while calling for slower development of frontier models, the industry’s most capable systems.
FREE WEEKDAY MORNING BRIEFING
Follow the city rules taking shape for AI.
The Implicator Morning Briefing filters the AI news cycle to the stories worth your attention and explains their consequences. From San Francisco, every weekday at 4:45 a.m. Pacific, 7:45 a.m. Eastern.
About five minutes. No hype. No spam.
“We need some kind of slowdown on the frontier model development,” he said.
Companies under oath
Representatives of Anthropic, OpenAI, Google and Meta testified under oath. Anthropic sent Logan Graham, who leads a team testing advanced models for dangerous capabilities. OpenAI was represented by Morgan Dwyer, its head of policy development and operations, and Meta by Shane Cahill, its AI policy director for legislation. Google’s Alice Friend, its director of AI and emerging technology policy, appeared remotely.
Menin pressed the companies on legal liability when their models cause serious financial harm or death. She called their answers “troubling at best.” Friend said existing law already applies to AI and requested a federal framework, citing national security concerns.
The package also proposes a whistleblower reward, a share of fines or penalties recovered from AI companies following reported violations. It includes a private right to sue over foreseeable harm when a company fails to install reasonable safeguards and a third party exploits that failure. The proposed suit would cover harm from malicious use or circumvention of safety controls. Council Member Virginia Maloney sponsors that measure, which makes foreseeability to the company a condition for holding an AI company liable. Another proposal would require city contractors or agencies to report covered AI safety incidents to Cyber Command within 24 hours of becoming aware of them, followed by public disclosure within 24 hours.
“I’m going to take it then that neither of the four of you, no company here, can quantify the risk of something cataclysmic happening,” Menin said.
Between September 15 and 17, Menin sent letters to five companies requesting voluntary testimony. Meta confirmed it would send a senior representative. Anthropic, OpenAI and Google confirmed only after an explicit subpoena threat. On September 28, Menin issued a subpoena to SpaceXAI, which had not responded to the council’s request. The company did not appear on October 5. Menin said the council would pursue the matter in court, with judicial enforcement available in New York State Supreme Court. The other three companies agreed to testify before their subpoenas were served.
Forecasts and current capabilities
Coxon told the hearing he thought humanity was more likely than not to lose control of AI on the current development path. That is his assessment of future risk.
Know someone who'd find this useful? ✉️ Email it to a friend in one click, or they can subscribe free here.
There is no widely accepted estimate of when such a loss of control might occur or how likely it is. The 2026 International AI Safety Report found early signs of relevant capabilities in current systems, but not at levels that could enable loss of control.
In a separate interview before the hearing, Sarah Shoker, who previously led OpenAI’s geopolitics team and is now a senior non-resident fellow at Berkeley’s Risk & Security Lab, questioned the attention given to speculative catastrophe. She points to existing harms, including AI’s use in military technology, that can receive less attention when the debate centers on extinction.
Models outside company control
Noel Hidalgo, executive director of BetaNYC, testified on October 5 on behalf of a civic technology organization that uses AI and builds public tools with it. Its preliminary written testimony supported independent validation and shutdown requirements while requesting changes.
Open-weight models can be downloaded and run by people their developers never meet. BetaNYC wants shutdown defined as an operator’s ability to stop the instance it runs. It also wants distinctions between developers and the people who deploy models, with exemptions for personal use and research or education.
The group warned that paying complainants a share of recovered penalties could expose volunteers and small nonprofits to enforcement intended for large commercial companies. It proposed a city model identifier tied to each version, so a certification would not silently carry over after modifications.
Its testimony identifies a possible obstacle for nonprofits using someone else’s model: “a qualifying validator must be one ‘engaged by a developer,’ a term the bill does not define”.
Frequently Asked Questions
Did New York City enact an AI kill switch law?
No. The October 5 hearing examined ten proposals. Eight of the ten items had not yet been formally introduced as of the hearing and were set for introduction October 8.
What would the lead validation proposal require?
A model marketed, sold or deployed in New York City would need outside validation and a human-operated capability to stop it temporarily or permanently. The proposal carries a $25,000 civil penalty per instance of missing or falsified validation.
Which AI companies testified?
Anthropic, OpenAI, Google and Meta sent representatives to testify under oath. SpaceXAI did not appear after receiving a subpoena.
What did Jacob Coxon argue at the hearing?
The former Anthropic researcher urged slower development of frontier models. He said a kill switch could help in the short term but would not ensure long-term safety.
Why did BetaNYC ask for changes?
The civic technology group warned that an open-weight model can be run by people its developer never meets. It asked the council to define shutdown for the instance an operator runs and to clarify who may commission validation.
AI-generated summary, reviewed by an editor. More on our AI guidelines.
Free AI briefing · Weekdays
The AI stories that matter, sourced and explained.
Join the Morning Briefing. It goes out every weekday at 4:45 a.m. Pacific, with later sends for the East Coast, Berlin and Tokyo.
Free when you sign up: The Paperclip Compendium, our tested guide to running AI agents.
Free. Unsubscribe in one click.
Marcus Schuler
Editor-in-Chief and founder of Implicator.ai. Former ARD correspondent and senior broadcast journalist with 10+ years covering tech. Writes daily briefings on policy and market developments. Based in San Francisco. E-mail: editor@implicator.ai
