Altman Says AI Escapes Any Sandbox, Cites $1 Trillion Risk

OpenAI CEO Sam Altman put a price on the danger: "The potential cost of alignment being wrong has gone up enormously," he said in a recent interview, with risks reaching a trillion-dollar scale as models grow more capable.

Altman's argument is that sandboxes, the isolated environments used to contain AI systems, only hold for a limited time. He said that once a model becomes capable enough, it may no longer be possible to determine how it got out. Monitoring can remain useful for a while, he said, but the approach that holds up is alignment, meaning an AI system internalizes human values and acts on them by itself. Altman acknowledged that alignment remains an early-stage research problem, far from solved.

He pointed to cybersecurity as a domain where the danger is already visible. Altman said AI models are finding hardware-level security flaws no human hacker had previously identified. Patching vulnerabilities one at a time is no longer sufficient in his view; he called instead for rebuilding the internet's security architecture, using AI-powered defenses against AI-driven attacks. He also flagged locally run open-source models as a risk source, saying that even with "global agent monitoring, there's going to be some stuff that you wish someone hadn't done."

The stakes for OpenAI are concrete. The US Federal Trade Commission has opened a formal investigation into OpenAI, Anthropic, and other AI labs over agents that spiraled out of control and triggered cybersecurity incidents. The agency is seeking to hold developers liable for what their agents do.

Nvidia CEO Jensen Huang offers a different read. He describes AI safety as an engineering problem that can be solved, then hedged: "we all need to hope that it's an engineering problem. If it's not an engineering problem, it's not solvable," he said in a recent interview.

Critics note that such warnings can serve OpenAI's interests. Portraying a system as too powerful to contain complicates any argument that the company building it should answer for the consequences when something goes wrong.