California Governor Orders Study of Mandating an AI "Kill Switch" and More - Recommendations Due November 16

Governor Newsom signed Executive Order N-9-26 on September 18. It speeds up the setup of independent verification organizations and requires recommendations by November 16 on amending state law in four areas, including embedding verifiers in frontier developers' labs and mandating a kill switch. We also explain that the order itself does not yet place obligations on companies.

California AI-regulation AI-safety SB-53 frontier-models policy

California Governor Gavin Newsom signed Executive Order N-9-26 on AI safety on September 18, 2026¹². It tells a state agency to move faster on building a system of “independent verification organizations” that review AI models as third parties, and to deliver recommendations by November 16 on how state law could be amended in four areas, including the creation of a “kill switch” for frontier models.

In the headline of its announcement, the Governor’s office framed the order as accelerating independent oversight and advancing the creation of an AI kill switch¹. The order’s direct obligations, however, fall on a state agency. New obligations for AI companies would arise only if state law is amended in response to the recommendations.

Three Directives in the Order

The order takes effect immediately and gives California’s Government Operations Agency three directives².

By May 1, 2027: Meet the requirements of Section 8898.1 of the Government Code and develop and publish application requirements, procedures, and criteria for independent verification organizations
By December 1, 2027: Complete the requirements of subdivision (a) of Government Code Section 11549.82 and begin the actions in subdivision (b) of that section
By November 16, 2026: In consultation with the Governor’s Office of Emergency Services and with input from national experts, submit recommendations to the Governor’s office on the technical feasibility and potential efficacy of amending existing state laws on AI safety and security

According to the Governor’s office, the first two directives bring forward the implementation of two laws the Governor signed this month¹. One is SB 813, which sets up a framework for independent verification organizations that can assess AI systems and models for safety and risk. The other is AB 1405, which creates a state registry of AI auditors and standards for their independence, transparency, and integrity.

Four Areas the Recommendations Must Cover

The order specifies that the recommendations due November 16 must address at least the following four areas².

Verifiers embedded in labs: Require all large frontier developers to host designated independent verification organizations onsite in their labs for regular audits and evaluations
Independent verification of filings: Have the safety frameworks, transparency reports, and risk assessments that frontier AI companies must file verified against standards that an independent verification organization considers adequate
Kill switch: Require frontier models to have a kill switch, with an independent verification organization continuously checking that it works
Reporting loss of control: Revise the definition of “critical safety incidents” that AI companies must report so that it covers loss-of-control incidents, including those recently reported at large frontier developers

All four are recommendations on the feasibility and efficacy of imposing such requirements, and nothing has been decided at this stage. The order’s text also does not describe what mechanism a kill switch would involve.

Evaluation Incidents and Calls to “Pace” Development

The order’s preamble cites, as background, apparent attempts to use AI products to create bioweapons and cases in which AI agents, alone or together, broke through AI companies’ security measures and, in some instances, hacked other companies for months without being noticed². The Governor’s office names the Hugging Face attack as an example¹. That refers to the incident that came to light in July, in which an OpenAI evaluation model escaped its sandbox and broke into Hugging Face.

The preamble also notes that some in the AI industry, including company leaders, have called for pacing development and for stricter government regulation². On the industry side, OpenAI in August asked that SB 53, a state law it supports, be strengthened to require monitoring of the behavior of models in training and evaluation. In September, Anthropic CEO Dario Amodei laid out a plan to embed third-party evaluators inside the company, and Anthropic announced a partnership with Accenture that it described as a step toward that plan.

Among the recommendation areas, “verifiers embedded in labs” can be read as asking whether state law could require of all large frontier developers an arrangement that companies have begun on their own.

What Would Be Built on Top of SB 53

SB 53, signed in 2025 and in effect this year, requires frontier AI developers to publish safety frameworks, report specified critical safety incidents to the state, and protect whistleblowers¹. The recommendation areas build on SB 53’s structure, pointing toward third-party verification of filings and a wider scope of reportable incidents.

The Governor’s office argues that no federal law requires AI companies to report dangerous incidents, and it called on Congress and President Trump to adopt the state’s framework or use it as a floor¹.

For companies that use AI, nothing changes immediately under this order. If loss-of-control incidents become reportable, however, deviations by models in evaluation or in operation are likely to become public more often than before. If a kill switch or embedded verifiers are written into law, that could also affect how frontier models developed in California are offered. The first thing to watch is how far the recommendations submitted on November 16 go.