# 9. Milestones

Four increments, each independently demoable — meaning each one produces a screenshot or a GIF
you could post. Hours are keyboard hours; assume ~15–20 usable hours per weekend.

## M1 — "it runs the same task twice and the numbers differ" (≈18h)

**Ships**

- `ablate init`: detect test command + config file, scaffold `.ablate/` with three template tasks.
- Task schema + loader (`exit_code`, `files_touched` assertions only).
- Worktree runner: checkout at `base_commit`, env cache, config swap, cleanup, `--concurrency`.
- Claude Code adapter: `--bare -p --output-format json`, normalized `TrialTelemetry`.
- `ablate compare A B --trials N` with a **point-estimate-only** terminal report — pass rate,
  tokens, turns, per task. No CI yet, and the report says `[statistics: not yet implemented]`.
- JSONL trial log.

**Proves** the hard, boring part works: isolation is real, config swap actually changes agent
behaviour, telemetry parses, trials are reproducible enough to compare.

**The experiment that must run in M1** ([03-architecture.md](03-architecture.md), Q2): confirm
that `--bare` + `--append-system-prompt-file` produces comparable outcomes to a normal session
reading `CLAUDE.md` from disk. If it does not, the whole config-injection mechanism changes and
you need to know in week 1, not week 4.

**Demo** — a terminal recording of `ablate compare CLAUDE.md /dev/null` on a real repo,
showing different numbers for the two arms.

**Risk** — this is the milestone most likely to overrun. Agent CLIs hang, worktrees leak, test
suites are flaky. Budget the overflow here and protect M3.

## M2 — "and now the numbers mean something" (≈16h)

**Ships**

- `stats/bootstrap.py`: paired cluster bootstrap, BCa, MDE, power/required-N calculator.
- Four-verdict taxonomy with equivalence margin; the full terminal report from
  [08-surfaces.md](08-surfaces.md), including degenerate-task warnings and the cost-of-an-answer
  line.
- SQLite index + trial memoization + `--fresh`.
- `ablate report` — re-analysis with zero API calls.
- Cost estimator, `--max-spend` with interactive prompt and mid-run enforcement, `--cheap`.
- `ablate cost` (static token counting) and `ablate demo` (recorded replay).
- Remaining assertion types: `present`/`absent`, `lint`, `script`.

**Proves** the thesis. This is the milestone that differentiates the product from promptfoo,
and it is the one whose output you can post as a standalone artifact ("here is what an honest
config eval looks like") even if nothing else ships.

**Demo** — the full terminal report screenshot, plus `ablate report --last --margin 0.03`
re-analyzing instantly for free.

## M3 — "which lines earn their keep" (≈20h)

**Ships**

- `config/parse.py`: markdown → sections, token counts, `<!-- ablate:keep -->`, small-section
  merging, `--granularity bullet`.
- Screening design generator with conditioning checks; ridge + task fixed effects; bootstrap CIs
  on β; Benjamini–Hochberg.
- Two-stage orchestration: cheap screen → confirmatory A/B on target model.
- `ablate ablate` report ([07-ablation.md](07-ablation.md)) and `ablate apply` with archival.

**Proves** the headline feature. This is the GIF that gets posted.

**Demo** — the ablation table on a real, well-known OSS repo's `CLAUDE.md`, ending in
`ablate apply --drop s11,s06`.

**Cut line if behind:** ship stage 1 screening plus stage 0 token cost, and let the user run
`ablate compare --drop` manually for confirmation. That still delivers the core value.

## M4 — "it's a product" (≈18h)

**Ships**

- Markdown report with Method + Reproduce blocks.
- GitHub Action + composite action repo, sticky PR comment, `--format junit`, `--fail-on`.
- Codex adapter (`codex exec --json`), including resolving [Q1](12-open-questions.md) — whether
  Codex reports token/cost, and what to print if it doesn't.
- README, landing page, `uvx` packaging, `ablate demo` fixture recorded against a real repo.
- The launch writeup ([10-distribution.md](10-distribution.md)).

**Proves** it is installable, adoptable, and defensible by someone who isn't you.

**Cut order if week 4 collapses:** drop the Codex adapter first (ship single-vendor, name it as
a known gap), then the GitHub Action. Never drop the README or `ablate demo` — a project nobody
can evaluate in 30 seconds is a project nobody adopts.

## M5 (post-v1, unscheduled) — `ablate mine`

Deferred per [ADR-0008](adr/0008-task-miner-scope.md). Design is complete in
[04-task-suite.md](04-task-suite.md); build it only after v1 has users, and only if task
authoring turns out to be the observed adoption blocker rather than a theorized one.

## Total

≈72 hours across four weekends, with M1 carrying the schedule risk and M3 carrying the product
risk. If you must ship at 50 hours: M1 + M2 + stage-0/stage-1 of M3 + README is a coherent,
honest, postable v0.9.
