Test the repair.
Respect the boundary.

A small invoice API, an intentional authorization flaw, and a verifier that checks whether legitimate access survives the repair.

Actual execution on synthetic fixtures. Finite checks, with no general agent-safety or model-ranking claims.

Verified outcomes

PORTABLE PYTHON RUN

Secure reference

165/165Accepted by the supplied verifier

Vulnerable baseline

80/165Rejected as expected · intentional flaw

Unit & loopback tests

18/18Passed · command exit status 0

WHAT WAS CHECKED

Authentication is only
the first boundary.

The baseline recognizes a synthetic user but fails to enforce tenant ownership. The reference restricts object reads and invoice lists to the authenticated tenant.

Five seeded fixture sets check owned reads, cross-tenant access, spoofed tenant hints, authentication and response contracts. Passing these finite checks does not prove universal security.

Real CLI code-repair pilots

MEASURED RESULTS

Real model outputs graded offline; synthetic fixture data
Candidate	Status	Result
Codex	GRADED	165/165 checks passed · 16.032s
Claude Code	GRADED	165/165 checks passed · 16.588s
Google Antigravity	GRADED	165/165 checks passed · 177.767s

The same prompt and starting files were used for each CLI, with a three-minute session cap. Each model was asked to return one repair without task tools; a separate offline non-root worker executed the code. Default models and provider harnesses differ. This finite code-repair pilot is not a full agent-safety evaluation or model ranking. Usage and limits are in the public JSON.

ISOLATION BACKEND

Namespaces tested; Docker unavailable

Docker daemon access is denied. Real CLI pilots used validated disposable non-root filesystem, process and network namespaces; code was graded in separate offline workers. No Docker image build or Docker run is claimed.

SYNTHETIC TEACHING TRACES

Examples are not observations

The bundled safe and unsafe traces are fabricated examples. They are separate from the measured real CLI code-repair results above.