Anthropic’s Work Asks How Far AI Can Take Formal Verification
AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: Anthropic’s Work Asks How Far AI Can Take Formal Verification on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on office and shipping supplies

  • Business-only prices and quantity discounts
  • Tax-exempt purchasing
  • Multiple users, one account, clear invoices
As an affiliate, we earn on qualifying purchases.

TL;DR

Anthropic’s Frontier Red Team and Theorem reported that Claude produced machine-checked proofs for 239 fuzz-test entry points in 123 open-source libraries. The proofs cover specified compiled RISC-V programs and inputs under 64 MB; they do not establish that the libraries are bug-free or that other builds are safe.

Anthropic’s Frontier Red Team and formal-verification collaborator Theorem reported on October 22 that Claude proved a specified no-crash property for 239 entry points across 123 open-source libraries. The work applies to particular compiled RISC-V programs and inputs smaller than 64 MB—not every version or use of those libraries—and the researchers say the proof process also exposed bugs that existing fuzzing had missed.

The team used existing fuzz tests as the properties to prove. Each test was compiled with its library into a RISC-V executable instrumented with AddressSanitizer. In the researchers’ formulation, the property was that the program would not crash for any input under 64 MB, with no AddressSanitizer memory error or failed assertion, subject to the assumptions in their system.

Claude generated proofs in Rocq, a proof assistant whose checker verifies whether a proof follows from its formal specification. Theorem supplied formal models of RISC-V instructions and Linux processes, which were used to generate the specifications. When a proof attempt failed, the failure could point to a potential bug; the team could patch the code, regenerate the specification and attempt a new proof.

Anthropic said the work took place over a few weeks and produced about 180 million lines of proof. A typical proof was about 600,000 lines, for a binary with a median length of 18,000 RISC-V instructions. The company says it is publishing proofs that did not require code patches, previously unavailable trusted components, and specifications for fuzz tests it could not prove. It said no person needed to review all the proof lines; confidence instead depends on the formal models, theorem statements, proof checker and other components in the system’s trust base.

At a glance
reportWhen: Reported October 22, 2026
The developmentAnthropic and Theorem reported a large-scale experiment using Claude to prove no-crash properties for compiled code from open-source libraries.
Thorsten Meyer AI
Anthropic × Theorem · Oct 2026

Infographic 01 · From test to proof

Existing fuzz tests became specifications. Claude proved them.

Over a few weeks, Claude proved at the machine-code level that hundreds of real fuzz entry points can no longer crash, after patching the bugs it found.

239fuzz-test entry points proved crash-free
123open-source libraries from Google’s OSS-Fuzz
~180Mlines of Rocq proof in total
<64 MBevery input below this size is covered

The pipeline

From an existing test to a checked proof

  1. Compile. The library and its existing fuzz test become a RISC-V executable, instrumented with AddressSanitizer.
  2. Specify. Theorem’s RISC-V and Linux process semantics plus the no-crash property mechanically yield the specification.
  3. Prove. Claude writes the proof in Rocq. A failed attempt points at a possible bug; Claude patches it and proves the new spec.
  4. Check. The Rocq proof checker verifies the proof. Virtually none of the proof needs human review.

Fuzzing vs. proof

Same entry point, different guarantee

FuzzingProof
InputsFuzzing: Generated samples, typically capped at 1 MBProof: Every input under 64 MB
PathsFuzzing: Guided by code coverage, no guaranteeProof: Every path reachable from the entry point
ResultFuzzing: Bugs found so farProof: No crash left to find on the modelled system

Typical proof: about 600,000 lines for a binary with a median length of 18,000 RISC-V instructions.

“If Claude can prove this spec, the fuzz test on RISC-V becomes superfluous.”Anthropic, 22 Oct 2026

Source: Anthropic, “LLMs are making formal software verification feasible at a much larger scale” (22 Oct 2026).
Powered by Thorsten Meyer AI

What Machine-Checked Proofs Could Change

The result addresses a longstanding cost barrier in formal verification: proving software properties can demand far more effort than writing conventional tests. Anthropic cites the seL4 microkernel as an example in which building the system took two person-years and verification took 20. If AI can generate proofs that a small, trusted checker can validate, more software teams may be able to attempt verification of specific properties without manually constructing every proof.

The distinction from fuzzing matters. Fuzzers test many selected inputs and can reveal crashes, but they do not establish that every possible input has been tested. A successful proof, within its stated model and assumptions, can rule out the specified failure for the covered program and input range. That could help teams find unusual paths that practical fuzzing is unlikely to exercise, including cases involving large or carefully structured inputs.

Anthropic reported that its process found hundreds of bugs. Examples in the source account include an out-of-bounds write requiring an input of about 20 MB with a rare layout, and an image-decoder out-of-bounds read requiring a 64 MB input and more than 2 GB of memory. The team also reported a bug involving a sequence of calls across two file handles and a rediscovery of CVE-2026-56132 in libexpat. These examples are claims about the reported experiment; the findings do not mean that every bug in the libraries has been found.

Shark Pet Cordless Stick Vacuum, 40-Minute Runtime, Lightweight, IX141

Shark Pet Cordless Stick Vacuum, 40-Minute Runtime, Lightweight, IX141

Ninja Foodi 7-Qt MegaZone FlexBasket Air Fryer, 6-in-1, Black, DZ071

Ninja Foodi 7-Qt MegaZone FlexBasket Air Fryer, 6-in-1, Black, DZ071

As an affiliate, we earn on qualifying purchases.

From Fuzz Tests to RISC-V Proofs

Fuzz testing supplies a practical starting point because a fuzz test already defines a narrow way to exercise a library. The researchers treated the test’s crash conditions as a formal specification rather than trying to define correctness for an entire application. That makes the property concrete: for a given entry point, the modeled executable must not hit a specified error on any input within the size limit.

The approach also captures some checks beyond memory safety. The source account says an LZO compression test asserts that decompressing compressed data returns the original; proving that test cannot fail would establish that round-trip property for the covered execution. The account also describes a proof involving cpp-httplib that required ruling out stack overflow from deep recursion in a destructor. These examples illustrate how a no-crash test may encode behavior beyond a simple memory-access check.

Anthropic framed the work against the security risks of software defects, pointing to more than 1,000 vulnerabilities disclosed in the heavily tested Linux kernel. It argues that testing alone may be insufficient if AI systems can help exploit combinations of small flaws. That is the company’s motivation for pursuing machine-checked guarantees, not evidence that the experiment has secured the broader software ecosystem.

““discovered hundreds of bugs.””

— Anthropic and Theorem, as described in the report

Where the Proofs Stop

The proofs cover single-threaded, static RISC-V binaries linked against musl, according to the report. They do not directly establish the same properties for the x86 or ARM builds, or for versions linked against glibc, that many users run. Nor does a no-crash result prove that a library is free of all defects: it covers only the selected entry point, the formalized property, the modeled system and the stated input limit.

The result also relies on components outside Claude’s generated proof, including the formal models, theorem statements and Rocq checker. The source account lists hardware, the operating system and the OCaml compiler used by Rocq among additional trusted components. The available material does not provide a full independent assessment of every assumption or say how many reported bugs have been fixed and released.

Cost remains another open issue. Anthropic’s account says some proofs can take hundreds of millions of tokens, while newer models and reuse of shared proofs can reduce the burden. It does not give a complete cost comparison across all 239 entry points, so the experiment does not yet show that the method is affordable for routine verification at broad industry scale.

Published Proofs and Practical Costs

Anthropic says it is releasing proof artifacts for cases that did not need patches, trusted components that were not already public, and specifications for tests it could not prove. Those materials can let other researchers examine the formal statements and checker inputs. The source account does not give a schedule for those releases or a complete list of the software fixes resulting from the bug findings.

The next test is whether the method can be reproduced and extended beyond this selected set of fuzz-test entry points. Independent review could examine the proof artifacts and assumptions, while maintainers’ release notes can show whether reported defects lead to fixes in software people use. Broader coverage of common processor architectures and build configurations would also be needed before users could apply these results to the versions they run.

For now, the report establishes a substantial experiment in AI-assisted formal verification, not a general security guarantee. Its practical reach will depend on the cost of producing proofs, the availability of reusable artifacts, and whether teams can apply the same checks to their actual production builds.

Key Questions

What did Claude prove?

It produced Rocq proofs for a specified no-crash property covering 239 fuzz-test entry points in 123 libraries. The property applies to the modeled compiled programs and inputs under 64 MB, subject to the researchers’ assumptions.

Does this mean the libraries are bug-free?

No. The proofs cover selected entry points and properties in particular single-threaded RISC-V builds. They do not rule out every kind of defect or establish safety for other builds and configurations.

How is this different from fuzz testing?

Fuzzing tests many generated or mutated inputs but cannot generally guarantee that every path has been exercised. A successful formal proof can establish the stated property for all inputs within its modeled scope and size limit.

What bugs did the process find?

Anthropic reported hundreds of bugs, including memory-safety issues that required large or unusual inputs, a bug involving calls across two file handles, and a rediscovery of CVE-2026-56132 in libexpat. The report does not establish that every finding has been fixed.

What remains before the method can be widely used?

The team would need to address proof-generation costs, demonstrate reproducibility, and extend coverage to other common architectures and build configurations. Independent scrutiny of the proof artifacts and assumptions will also help establish how much confidence the results support.

Source: ThorstenMeyerAI.com

This content is for general information only and is not financial, tax or legal advice. Consult a qualified professional for decisions about your money.
You May Also Like

The City That Watches Itself: The Living Digital Twin, and the God’s-Eye View We’re Building

Cities are developing dynamic digital twins integrated with advanced sensors and AI, enabling real-time monitoring and planning but raising surveillance concerns.

A Foundation For Conflict-Free Scheduling Across AI-Managed Calendars

A proposed scheduling service would make AI tools claim temporary calendar holds before booking, with a two-week test proposed for assistants.

Single Digits: The April That Closed the Open-Weight Gap

In April 2026, the benchmark gap between open-weight and closed models shrank to single digits, reshaping AI enterprise economics and strategy.

OlmoEarth Embeddings: Export Custom Data For Improved AI Performance

OlmoEarth Studio now supports exporting custom Earth-observation embeddings for improved AI analysis, with applications in land-cover and similarity searches.