Quickstart

Test Evidence checks a narrow, specific claim: that a change’s tests actually ran and passed, that the test surface wasn’t quietly shrunk, and that a kept test wasn’t gutted so it can never fail again. This quickstart wires the pieces together and gets you to your first report in about five minutes.

1. Install and sign in

If you don’t already have the CLI, install it and sign in; see Installation for the full walkthrough.

curl -fsSL https://dl.shipmoor.dev/install.sh | bash
shipmoor login

test-evidence is gated behind its own test_evidence entitlement, in the Shipmoor IC plan alongside claim_check. Confirm it’s enabled:

shipmoor capabilities --json   # .capabilities.test_evidence.enabled is true or false

The shipmoor test-evidence hook subcommand you wire in the next step is never gated. It always runs, whether you’re on Community or IC.

2. Wire the PostToolUse hook

The strongest form of the passed signal comes from a harvested result bound to the exact working-tree state your agent tested, not a fresh re-run. That harvesting depends on a lifecycle hook that watches your coding agent’s own test runs as they happen.

For Claude Code, add this to .claude/settings.json (project or user level):

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Bash",
        "hooks": [{ "type": "command", "command": "shipmoor test-evidence hook" }]
      }
    ]
  }
}

Every time your agent runs a Bash command, Claude Code pipes the tool-result event to shipmoor test-evidence hook on stdin. If the command looks like a real test run, it’s recorded to a local evidence spool at .shipmoor/test-evidence/spool/. Anything else, a non-test command, or a directory that isn’t a git repo, is a silent no-op. The hook always exits 0: a lifecycle hook must never interrupt an agent’s turn, gated feature or not.

3. Run it on a real change

Make a small change, or let your agent make one, for example a bug fix with a test attached, and let the agent run its own test suite the way it normally would. With the hook wired, that run is already recorded. Now check it:

shipmoor test-evidence

With no target and no selector, shipmoor test-evidence looks at your working tree, the same default every Shipmoor command uses. Narrow it with --staged, --diff <spec>, --all, or a committed --from/--to range; see the CLI reference for all of them.

4. Read the report

A clean run over a change that added a test and didn’t touch anything else reads like this:

Test Evidence  ·  exit 0
  passed    ✓ 24/24 tests passed  (harvested: hook spool, bound to working tree)
  reduced   ✓ no tests removed
  weakened  ○ not judged (no --agent configured)

• passed: grounded in the harvested spool entry, not a re-run and not a trusted exit code. It names its own basis, here hook spool, so you know exactly what the claim rests on.
• reduced: deterministic, and clean here because nothing disappeared between base and head.
• weakened: no --agent was passed, so the signal honestly discloses that it wasn’t judged rather than guessing. Even with an agent configured, this line can only ever read weakened or stay silent; it has no way to certify a test as fine. See the overview for why.

If a test had silently disappeared instead, reduced would read ✗ 1 test removed since base (no removal on record) and the command would exit non-zero. A disclosed, reasoned removal recorded in .shipmoor/test-removals is the honest way to clear it. See Signals & anti-gaming for exactly how each signal is built to resist being gamed.

5. Optional: arm the weakened judge

passed and reduced are fully deterministic and never need a model. weakened is different: catching a gutted assertion has no mechanical rule, so it rides the same judge machinery Claim Check uses. Pass an agent to turn it on:

shipmoor test-evidence --agent claude

On a change that didn’t touch a kept test’s assertions, the line simply reads weakened ○ nothing flagged, the same honest silence as before, just backed now by an actual judgment instead of an abstention. On a change where an assertion was hollowed out, the same line becomes a disclosed finding instead, for example weakened ✗ assertion gutted at tests/test_pricing.py:14 (assert True in place of a real check). Notice what never appears next to weakened: a checkmark. The signal can flag a problem or say nothing. It never certifies a test as fine.

The --agent value is a subprocess you supply, the same bring-your-own-model approach as Claim Check’s judge; Shipmoor calls no model of its own here either.

Next

• CLI reference: every flag, selector, and exit code.
• Signals & anti-gaming: the full depth on passed, reduced, and weakened, and how each is built to resist being gamed.
• Using Test Evidence with Claim Check: how these signals compose into a claim-check run automatically.
• Overview: the problem it solves and the binder-not-oracle design.