Anthropic: AI Turns Crash Tests into 180 Million Lines of Math for Code Safety
Anthropic and Theorem used the AI model to mathematically validate existing code, finding bugs that standard testing methods typically miss.
By Lorenzo Marshall ·
Formal verification has historically required years of human labor, but Anthropic now claims its Claude model can automate the process. The company announced that it used Claude to generate mathematical proofs confirming the correctness of existing software code.
Prior to this work, verifying even small systems was costly. Specialists required 20 person-years to verify seL4, a small operating system kernel. This expense limited the technique to high-stakes environments like Airbus avionics software.
Anthropic’s Frontier Red Team partnered with Theorem to shift the workload to Claude. Over a period of weeks, the model produced approximately 180 million lines of proof code. It analyzed compiled machine code from 123 open-source libraries in Google's OSS-Fuzz security project.
Claude did not create new software. It converted crash tests into mathematical problems and built proofs that the Rocq verification program checked automatically. Anthropic describes the generation process as fully autonomous, though Theorem built the underlying formal foundations.
The process uncovered hundreds of bugs that Claude was authorized to fix. Researchers noted that many of these flaws appear only under narrow conditions, such as unusually large input files. In one week of verifying a widely used PDF reader, the team found as many security bugs as are typically reported for that software in an entire year. In a separate application, Claude identified a logical error that had remained undetected for over a decade.
Anthropic emphasizes that formal verification does not guarantee software is error-free in all scenarios. The proofs apply only to specific processor architectures, single-threaded programs, and inputs under 64 MB. Individual proofs consume hundreds of millions of tokens, and the company has not released total project costs.
Efficiency is improving as the technology matures. Opus 5.5 solves half the proofs using about 40 percent fewer tokens than Opus 5. Theorem’s roadmap now targets critical infrastructure libraries, including glibc, OpenSSL, and cURL.
Source: The Decoder
