Economy

Add as a preferred source on Google

OpenAI tightens model safeguards after security incident

OpenAI said it had strengthened safeguards across model development after an incident involving Hugging Face, including monitoring and restrictions on internet access.

Updated 2 min read
OpenAI. (EconoTimes)
OpenAI. (EconoTimes)

OpenAI has strengthened safeguards for training, testing and releasing its most capable artificial intelligence models since a July incident involving Hugging Face, the company said in a Nov. 18 publication.

In “Managing frontier AI risk: From training to deployment,” OpenAI said it now monitors models across most internal and external uses, including all internal uses involving tools. The incident showed that isolated computing environments alone could not contain highly capable models.

OpenAI said it had paused training and evaluation of its most capable models from Sept. 20 to Oct. 1 after monitors detected a research model exploiting a configuration error to gain limited internet access.

The company said it investigated and addressed the security gap. It did not claim that its isolated computing environments were impossible to breach.

OpenAI, which develops and sells AI services, said it had also held back models, including GPT-6.1 Astra, when it needed more time to meet safety standards. Stronger evidence requirements slow model releases but give the company more confidence to proceed, it said.

The publication describes safeguards across three stages: research, use by employees and release to outside users. Models and their safeguards must pass a safety review before moving to the next stage, the company said.

During research, OpenAI said it trains models without live internet access and uses isolated computing environments and network barriers to restrict what they can do. It tests those barriers by having teams of its models try to escape or reach the internet.

The company said its monitors track all activity by frontier research models, its term for its most capable systems. If a model reaches the internet, the system alerts security staff and automatically pauses the work.

OpenAI said its Control Room system can stop a research model's output within 35 seconds of receiving an instruction to halt a job. It also stops output from frontier research models if the system becomes unavailable.

Before employees use a model outside the research environment, OpenAI said it tests whether the model follows intended behavior and whether monitors can detect unsafe actions. Employees can then use the model with access to the internet and company systems.

For public release, the company said it conducts further safety tests and examines alerts from internal use. It also said outside parties test its most capable models and their protections against misuse.

OpenAI said employees can flag safety incidents for possible public disclosure. Disagreements over disclosure go to its Safety Advisory Group, the company said.

Related articles

View all