# 2. Scope

Budget: one solo maintainer, ~4 weekends (~80 hours of real keyboard time, assume 60 after
life happens). Constraint: usable in under five minutes, no signup.

## v1 feature list

Ordered by load-bearingness. If you run out of time, cut from the bottom.

1. **Trial runner.** N isolated trials per (task, config) pair. Each trial gets a clean `git
   worktree` checkout, the config swapped in, the agent invoked headlessly, and the resulting
   diff + trace captured. Parallel across trials with a configurable concurrency cap.
2. **Claude Code adapter.** `claude --bare -p ... --output-format json`. `--bare` is
   load-bearing: it skips auto-discovery of hooks, plugins, MCP servers, and `CLAUDE.md`, which
   is the only way to guarantee that the config under test is the *only* config in play.
3. **Deterministic assertion vocabulary.** Test-suite exit code, files-touched set, regex
   present/absent, lint rule, custom script. No LLM judge ([ADR-0004](adr/0004-no-llm-judge.md)).
4. **A/B report with a real confidence interval.** Paired cluster bootstrap over tasks. Primary
   metric = tokens/turns per task; secondary = pass rate. Explicit "underpowered / indistinguish-
   able from noise" verdict with the power calculation attached
   ([05-statistics.md](05-statistics.md)).
5. **Section-level ablation.** Parse config into sections, run a randomized screening design,
   attribute effect per section, output a keep/cut recommendation with per-section token cost
   ([07-ablation.md](07-ablation.md)).
6. **Cost controls.** Pre-run estimate, hard spend ceiling with a confirm prompt, response
   caching, `--cheap` mode, incremental re-runs that reuse cached trials.
7. **Result store + re-analysis.** Every trial persisted; `ablate report` re-analyzes without
   re-running a single agent call.
8. **Terminal + markdown reports.** ([08-surfaces.md](08-surfaces.md))
9. **`ablate init`.** Scaffolds a starter task suite by detecting the project's test command and
   generating 3 template tasks. This is the five-minute promise.
10. **GitHub Action.** Fails a PR when a config change regresses the guardrail metric.
11. **Codex adapter.** `codex exec --json`. Second vendor is the anti-displacement moat, but it
    is genuinely second priority to getting one vendor right.

## Non-goals for v1 — explicit

- **No hosted service, no accounts, no telemetry.** Not even opt-in. The moment you have a
  server you have a second product.
- **No web UI.** Terminal + markdown. A local viewer is a v2 nice-to-have.
- **No LLM-as-judge.** ([ADR-0004](adr/0004-no-llm-judge.md))
- **No config *generation* or auto-fix.** You report; the human edits. Generating config puts
  you in a crowded, low-trust space and undermines the "measurement instrument" positioning.
- **No support for Cursor, Windsurf, Aider, or opencode.** `.cursorrules` is in the pitch but
  Cursor has no documented headless invocation I could verify. You can *test a `.cursorrules`
  file* by feeding it to Claude Code or Codex as the config, which is a legitimate thing to do,
  but do not claim Cursor support.
- **No multi-repo / monorepo-slice orchestration.**
- **No cross-model comparison** ("is Sonnet or Opus better for my repo"). Adjacent, tempting,
  and a completely different product.
- **No trace-level assertions** (assert on tool-call sequences). promptfoo has these; they are
  brittle across agent versions and expensive to maintain.
- **No `skills/` directory ablation in v1.** Same machinery eventually, but the parse unit is
  different (whole files, with frontmatter and progressive disclosure) and it doubles the config
  model. v1 ablates one markdown file.

## Things I am arguing you down from

### Cut: the automated git-history task miner

You asked for a miner that picks a merged PR, reverts it, feeds the agent the issue body, and
compares the produced diff to the real one. It is the most intellectually interesting part of the
plan and the most likely to eat two of your four weekends and ship brittle.

The hard parts, none of which are optional for it to work:

- **Filtering.** Most merged PRs make bad tasks: no tests, multi-concern, dependent on
  unmerged state, unrunnable environments, or an issue body that doesn't specify the change.
  A useful filter is a nontrivial classifier.
- **Environment reproduction.** Reverting a PR from 8 months ago and getting the test suite to
  run requires the historical dependency state. This is precisely what SWE-bench spends its
  entire Docker layer cache on.
- **Similarity scoring.** Diff similarity is either brittle (exact match) or requires a judge
  (which you've excluded), and "same behavior, different implementation" is the common case.

**Recommendation:** ship `ablate mine` as an explicitly experimental command that *proposes*
candidate tasks — it filters PRs, generates a draft task file per candidate, and asks the human
to review and keep. It never auto-generates a suite you then trust. The design is fully
specified in [04-task-suite.md](04-task-suite.md) so you can build it in week 5 if v1 lands;
[ADR-0008](adr/0008-task-miner-scope.md) records the call.

The reason this is safe to defer: **the task suite is not your differentiator.** Anyone can write
five tasks against their own repo in twenty minutes. The differentiator is what you do with the
results. Do not spend your scarce weekends on task acquisition.

### Cut: the `.cursorrules` / multi-format promise

One config file format (markdown with headings) covers `CLAUDE.md`, `AGENTS.md`, and
`.cursorrules` — they are all markdown. Support "a markdown config file at a path you specify."
Do not enumerate vendors in the docs; it creates support obligations you can't meet.

### Cut: parallelism beyond a simple semaphore

Trials are IO-bound on the model API. A `ThreadPoolExecutor` with `--concurrency` (default 4)
is enough. Do not build a job queue.

### Keep, despite the cost: the Codex adapter

I would normally cut a second vendor from a four-weekend v1. Keep it, because it is the single
clearest structural defense against `skill-creator` expanding into your space, and because it
forces the agent abstraction to be real rather than a Claude Code wrapper with aspirations.
Budget it at milestone 4 and accept shipping without it if week 4 goes badly.

## The one feature that, if cut, kills the product

**Section-level ablation attribution** (#5).

Without it, `ablate` is "promptfoo with confidence intervals" — a real improvement, not a reason
to install a new tool. Ablation is the only thing that answers the question the user actually
has, which is not "is config A better than config B" (they can guess) but **"which of these 400
lines can I delete."**

The nuance worth naming: ablation is why someone *installs* it; the statistics are why they
*believe* it. Ship ablation with sloppy statistics and you're a random-number generator with
good UX. Ship statistics with no ablation and you're a library. Both are needed, but if a knife
is at your throat in week 4, ship ablation on a smaller task suite rather than dropping ablation
to polish the CI.
