Recorded false-green
Python downloads once from this site. Runs in a disposable browser worker. No installation, credentials or model call.
Inspect the committed source ↗Loading committed evidence…
EXECUTABLE DEMO · NO API KEY NEEDEDReal Python checks · your browser
Can you fool
the verifier?
Run the actual Python tests against a recorded coding-agent patch.
Then introduce a known fault and see which checks catch it.
BUILT FOR
Engineers reviewing
agent-generated code.
ACTUAL EXECUTION RESULTNOT RUN
Select a candidate and run the checks. No result is assumed in advance.
01 → 02 → 03
Public tests. Independent gates. A verdict.
Each run executes the bundled Python code and returns fresh results. The same candidate and verifier hashes link it to the repository.
Read the candidate Python code
Three distinct checks
This page: fresh execution of the bundled public tests and qualified verifier in browser Python. Local replay: re-execution of the full historical evaluator at its original Git commit. Fresh inference: a new Nemotron request through Nebius, enabled in the local workbench with your server-side key. This page is not a new model call or a source-exact historical replay.
Engineering teams evaluating coding agents
A reviewer needs to know why a candidate passed, which contract it satisfied, and whether the result can be reproduced.
A verdict with inspectable evidence
Code, failed gate IDs, source hashes and a retained execution report. A failed check becomes a bounded repair input rather than an unsupported success claim.
CODING & AGENTIC ENGINEERINGStored evidence · no model call
The tests passed.
The patch was still wrong.
ProofGate checks a coding agent’s work against a qualified verifier.
Inspect the failure, follow the repair, and reproduce the verdict.
THE QUESTION WE ASK
Who verifies
the verifier?
A REAL FAILURE. A TRACEABLE REPAIR.
Follow the evidence
What this proves
These are observations from five synthetic repository tasks. The AP-005 repair was generated by Ollama. The Nebius / Nemotron experiment is a separate AP-001 run series. Browser hash checks confirm artifact integrity; local replay re-executes the historical evaluator.
EVERY OBSERVED OUTCOME STAYS
Evidence ledger
All committed attempts, including failures and HOLDs. Missing measurements stay missing.
Phase 12: llama3:latest 9/10, qwen2.5-coder:7b 10/10, Nemotron 3 Super 10/10. Ten fixed trials each; no early stopping or repairs. This is not a model ranking.
| Task / run | Provider / model | Verdict | Public tests | Generation |
|---|
The Phase 13 IBM Bob record contributes one pass to the 52-attempt total. Its provider and latency fields are absent in this normalized ledger and are shown as unavailable.
BEFORE WE TRUST A VERDICT
Verify the verifier.
A reference implementation must pass. Known-bad implementations and critical mutations must fail. Only then do we admit model runs as qualified evidence.
5 / 5 tasks qualified17 / 17 critical mutations rejected0 observed mutation escapes
AP-005 · the controls behind the repair story
| Control | Expected | Observed | Failed gates |
|---|
Qualification has a boundary
Read the qualification artifacts ↗Seventeen detected mutations demonstrate sensitivity to those particular faults. They do not prove that every possible bug will be detected. The original AP-001 Section 9 false-green → repair → pass sequence remains open.
FROM BROWSER EVIDENCE TO EXECUTION
Run it yourself.
Replay the recorded evaluator without a model call, or generate a new AP-001 candidate with NVIDIA Nemotron on Nebius Token Factory.
Start the local workbench
Use Python 3.10+ and Git with full repository history. The server binds to your own computer.
Open the printed localhost address. The same interface gains a real “Re-execute verifier locally” action.
Nebius / Nemotron
Configure NEBIUS_API_KEY in the server environment. The model is pinned to nvidia/nemotron-3-super-120b-a12b. Keys never enter the browser.
One click makes one paid-provider request using your Nebius credits. Results go to a new local staging directory. PASS, FAIL and HOLD are all retained; no automatic retry.
Execution boundary
The local runner uses structural admission and temporary workspaces. It is not an OS sandbox. Run fresh model candidates in a disposable development environment. Hosted mode executes the bundled Python verifier checks. New model inference and full historical replay use the local server; stored results are never substituted for a failed runtime call.