pi /until-done in practice: the agent works until the proof lands

The core problem of every autonomous coding agent is not speed, it is honesty: the agent reports "done" as soon as it believes it is done, and tests that are still red are easy to overlook. /until-done for the pi coding agent inverts the trust model. The end of a task is not decided by the model but by a pre-defined, machine-verifiable proof — and even that proof is countersigned by a second LLM, the "judge", before the goal can be marked done.

The pattern is known as a "Ralph loop" (Hermes Agent's /goal, the Ralph loop in Codex CLI). pi-until-done is the native Pi implementation: an extension that turns an intent into a locked contract (the "North Star"), a dependency-aware plan, and a budgeted autonomy loop. This article covers the installation, a full run, and the commands you need day to day.

Installation and prerequisites

pi install npm:pi-until-done

Git and local sources work too (pi install github:srinitude/pi-until-done, pi -e /path/to/pi-until-done/extensions/until-done.ts). Pi loads the extension on the next start; verify with Ctrl+O or pi list.

Three prerequisites are worth checking first:

Prerequisite	Why it matters
Pi version lockstep	The extension pins an exact Pi release — as of 0.3.1 (July 2026): @earendil-works/pi-*@0.81.1. We installed and loaded it on the current Pi 1.0.2 anyway: no errors, and the --until-done flag path fires (identical behaviour on 0.99.2). Officially only 0.81.1 is CI-tested — after a pi update --self, run pi update --extensions and check for a matching release.
Node >=22.19	Runtime requirement of the extension. The bundled Pi distribution ships Node 22.23.1, so this is satisfied.
mise on PATH	Verification commands run through mise exec --; a command that already starts with mise run/mise exec is preserved as-is.

Tested (October 2026): pi install npm:pi-until-done succeeded on Pi 1.0.2 (and for comparison 0.99.2). The extension loaded cleanly across several sessions — also alongside other installed packages — and the --until-done flag path reached the setup logic. The documented full loop including the judge round-trip is only CI-tested against 0.81.1 — if you plan long productive runs on a newer Pi, run one complete cycle with your intended target model before relying on it.

The first run

A task starts with an intent — ideally one you can translate into a verifiable end state:

/until-done finish migrating the auth test suite from Jest to Vitest

Pi then runs through a fixed sequence:

1. Clarify – outcome, goal type (ticket vs. exploratory), reachable evidence surfaces (logs, tests, builds), and the verifyCommand.
2. Show the contract – goal, "done when" criteria, verifyCommand, and askBefore boundaries are presented.
3. Approve – confirm with /until-done approve; only now is the North Star locked via until_done_set. The five fields — goal, done criteria, verifyCommand, askBefore, judge mode — are immutable for the lifetime of the goal.
4. Work – Pi processes the phases: ANALYSIS → BOOTSTRAP → RED → GREEN → REFACTOR → VERIFY → DONE. For code goals this is TDD-first: before any code change comes a red test task (RED), and the run ends with verification.
5. Continue – after each agent_settled, the extension resumes autonomously while the goal is active. A hidden re-anchor entry after session_compact keeps the loop alive across compactions.
6. Prove – when all tasks are done, Pi calls until_done_complete with the verbatim output of the verifyCommand as evidence.
7. Adjudicate – the judge checks goal, criteria, and cited evidence and returns { "verdict": "done" | "continue" }. Only a done verdict closes the goal. Exports land in .until-done/tasks.yaml and .until-done/distilled.md.

A realistic contract for the example above would look like this:

goal: Migrate the auth test suite from Jest to Vitest
doneCriteria:
  - "all tests in packages/auth pass under vitest"
  - "no jest references left in packages/auth"
verifyCommand: pnpm vitest run packages/auth
askBefore: ["git push", "git commit -m 'migrate auth to vitest'"]
judgeModel: { provider: "anthropic", modelId: "claude-sonnet-4-5" }

The judge: why a second model

The most common failure of the Ralph loop is self-evaluation: a model grading its own work will readily talk itself into a premature "done". pi-until-done therefore always requires a judge — there is no silent way past it:

• Cross-model (recommended): a model different from the one executing, e.g. Sonnet executing and Opus judging. The judge sees only the goal, criteria, verifyCommand, and the cited evidence — no executor history, no bias.
• Same-model: sameModelJudge: true — deliberately weaker, for when no second model is available.
• The judge returns strict JSON; a continue verdict blocks completion and states the gap.

Configure the session default once:

/until-done judge anthropic/claude-sonnet-4-5
/until-done judge same
/until-done judge clear

Important: the judge receives the completion evidence. Do not put secrets in the goal, evidence, task notes, or exports.

Command reference

Command	Purpose
/until-done <intent>	Draft and display a new goal contract + plan
/until-done approve	Open the approval dialog for the visible draft
/until-done status	One-line current state
/until-done detail	Full contract + state
/until-done tasks	Current plan (YAML)
/until-done plan	Path to .until-done/tasks.yaml
/until-done pause / resume	Halt the loop (goal kept) / resume (re-armed)
/until-done cancel	Permanently clear the goal (only way to change the North Star)
/until-done budget N	Set the turn budget (default 20, confirm dialog above 500)
/until-done autopilot	Skip the setup dialog for this session
/until-done judge …	Show/set/clear the judge default
/until-done ask <question>	Ask a side question without interrupting the loop
/until-done northstar	Show the locked contract fields
/until-done replan-log	All plan revisions with reasons
/until-done help	Built-in help

Practical tips

• Falsifiable done criteria. "Tests green" plus "no X left" beats "neatly refactored". The judge can check literal findings, not moods.
• The verifyCommand is the law. Not "tests pass", but pnpm vitest run … with the output quoted. The judge rejects paraphrases — retrying judge_rejected with the same evidence is pointless.
• Set budgets. /until-done budget 30 as needed. The loop has several brakes: turn budget, a spin guard on repeated identical failures, user interjections, and a clean-end nudge when all tasks are done but complete is still pending. After two such nudges the loop pauses and waits for you.
• Side work via ask. /until-done ask Is the NPM token in place yet? answers interleaved questions without touching the goal state.
• Replan instead of guessing. If new work surfaces, Pi calls until_done_replan with a reason (residual_work_discovered) — it does not quietly invent more work.

When not to use it

• One-shot tasks. A single edit or a question needs no loop and no judge — every round costs at least one additional judge call.
• Open goals without a checkable end state. "Make the user happy" and "research X" cannot be measured against a falsifiable criterion. For documentation/research goals use the exploratory/none goal type — or work interactively as usual.
• When no second model is available and same-model self-judging feels too weak. The judge is the core of an honest completion.

Placement

/until-done is part of a wider family of loop tools. untilgreen is the counterpoint: deterministic shell gates decide completion instead of an LLM signature; loop-until (Codex), do-until/cc-until-done (Claude Code) and the goal extensions (@narumitw/pi-goal, pi-goal-x) cover similar patterns. For the loop category's place in the overall ecosystem — including untilgreen, where an exit code rather than a model decides "done" — see the Pi add-on overview.

The value of /until-done is the burden of proof: the agent may not exit until a second model has countersigned the verbatim output of the verification command. For long autonomous runs you gain exactly the property coding agents most often lack — a traceable, guarded completion. Start small: one goal, one verifyCommand, one budget — and after the first green verdict, the agent's self-confidence suddenly looks a lot smaller.