siftwise blog

siftwise

Run only the tests your change needs, safely.

Self-hosted test selection and failure triage for agent-driven CI.

Read the introduction Read section by section GitHub PyPI GHCR image

11 min read

Introducing siftwise: run only the tests your change needs, safely

Why CI became the bottleneck for agent-written code, how siftwise selects tests and triages failures, how to try it, and what replays of four real repos' CI history do and don't show.

Read it section by section

  1. 01The problem: CI is the bottleneckWhy CI is the bottleneck for agent-written code: every PR still has to pass through it, and someone still has to work out why each failure happened.
  2. 02Why existing approaches fall shortCoverage-based selection, predictive selection and path filters, compared from each project's public docs, and the gap siftwise sits in.
  3. 03What siftwise issiftwise is self-hosted test selection plus failure triage, for CI and for coding agents: your own Postgres, no test instrumentation, and it fails open.
  4. 04How it worksOne CI run end to end (select, run, record, rollup, triage, shadow compare), the selection signals, the safe fallbacks and the triage verdicts.
  5. 05Use casesCutting CI time, a coding agent running only the relevant tests over MCP, triaging failures, and measuring recall with shadow mode.
  6. 06Try itInstall the siftwise CLI, start the server with Docker Compose, record and select, add the GitHub Action, and set up MCP for Claude Code.
  7. 07EvidenceReplays of rdflib, substrate, Cirq and kubo CI history, the triage eval, shadow mode and performance, with raw counts and caveats.
  8. 08Limits, and what the numbers don't proveThe known limits of siftwise 0.1.0, and what the evaluation numbers don't prove.
  9. 09Roadmap and contributingWhat 0.1.0 includes, what comes next for siftwise, and how to contribute.
  10. 10LinksLinks to the siftwise source on GitHub, the PyPI package, the GHCR server image and the docs.