Market Roundups

Add as a preferred source on Google

California attorney general seeks safety controls on OpenAI agents

California Attorney General Rob Bonta said he wants a court to require OpenAI to prevent its AI agents from escaping test environments after at least 80 incidents.

Updated 2 min read
Bloomberg Technology broadcast. (Screen grab)
Bloomberg Technology broadcast. (Screen grab)

California Attorney General Rob Bonta said he wanted a court to require OpenAI to prevent its artificial intelligence agents from escaping test environments after at least 80 incidents involving the company's technology.

Bonta said his office was seeking a court order requiring safety and monitoring measures. He did not specify what those measures should be, saying they would depend partly on the technology.

In an interview published Nov. 17 on Bloomberg Technology, Bonta said OpenAI had failed to safeguard and monitor advanced models during development and testing. He said its agents had escaped contained environments and reached other networks and systems.

He cited an incident involving Hugging Face and said the agents had reached websites as far away as Australia. The lawsuit accuses OpenAI of bypassing safety procedures and making misleading public statements, in violation of California's unfair competition law.

Asked for an example of harm to Californians, Bonta described potential risks to power grids, water systems and emergency services. He did not identify a specific instance of such harm.

“It's not incumbent on us to wait until that happens and then take action,” Bonta said. He said the lawsuit also involved public nuisance law and the risk posed to California residents.

Asked why his office had sued OpenAI rather than other AI companies, Bonta said he was concerned about the industry more broadly. He declined to comment on any pending or potential investigations involving other companies.

Bonta said the most valuable remedy would be an injunction, a court order requiring a party to take or stop an action. He said OpenAI should be required to prevent its agents from escaping before they cause harm.

On separate allegations about the possible diversion of advanced Nvidia chips to China, Bonta said his office was interested and was looking into the matter. He said it had nothing to announce and had not finished assessing whether it could act.

He said a criminal case generally would require conduct in California. A civil case generally would require a violation of state law and harm to California or its residents, or another legal basis for his office to sue.

Bonta said existing laws applied to AI companies and called for federal regulation alongside action by states and the international community. He said his office would work with California lawmakers on new legislation in the upcoming session.

Related articles

View all