Blog · The Agentic CTO — part 0 · the foundation
Build the harness first
I don't read the code any more. That isn't trust in the model — it's the harness. This is the part that has to exist before anything in parts 1–6 works.
A CTO asked me last week how I can merge nearly 700 pull requests a month and not read the code. He'd heard people say they never look at the code, and he wanted to know whether that was bravado or a method.
It's a method, and the model is the smallest part of it. The industry's shorthand is agent = model + harness: the model you rent, the harness you own. My harness is why a PR from an agent at 3am and a PR from me on a Tuesday face the same nine gates — and why I can afford not to be one of them.
Parts 1 to 6 assume this layer exists. They describe how the org runs on top of it. This is Part 0 because it comes first, and because it's the article I'd want handed to me before touching any of the rest: names, files, numbers.
The stack, named on purpose
Part 1 says the product names are this quarter's stack, not the architecture, and that's still true. But you asked, so here it is as of September 2026.
Models. Claude Code on the Max plan is the operator. Opus does roughly three quarters of the work. The hardest quarter — design synthesis, ledger migrations, anything where a wrong assumption costs a day — goes to Fable. Sonnet does the volume: sub-agents spawned by the planner, and the QA driver that opens a browser. The rule of thumb: the model choosing what to do is never cheaper than the model doing it.
Review. Cursor's Bugbot reviews every PR and is a required check. That's not enough on its own — a review comment is not a gate — so a second required check, bugbot-triage, fails the PR while any High finding sits unanswered. The web merge button once merged an unresolved High plus two Mediums through fully green CI. It can't now.
Everything else is boring on purpose. GitHub Actions for CI and deploys. Bun as runtime and bun:test for unit tests. Playwright for end-to-end. mise as the single task runner, because an agent that can call mise run merge never needs to know the four commands behind it. OpenTofu for infrastructure, applied by a merged PR and nothing else.
The contract: one file, read every turn
Every agent — Claude Code, Codex, Cursor, whichever — reads AGENTS.md on every turn. It's 181 lines and I fight to keep it that size. CLAUDE.md is a symlink to it, so there is exactly one contract.
The top says what the file is for:
# AGENTS
Slim, always-on contract every agent reads on every turn. Keep only invariants
that affect nearly every task here; repo-specific playbooks live in focused
`herita-*` skills under `.agents/skills/`.
Then the stack, a source map, and the hard rules. The rules that hold are the ones written the week something went wrong, with the date left in:
- **A check that has never executed is not evidence, however green it looks.**
On 2026-09-01 five instances surfaced in one day, each invisible until
something forced execution: three ledger specs the release gate could not
run at all (no TigerBeetle tunnel) and so had gated every release by
skipping; … a `qa-agent` "pass" whose suite had silently excluded the specs
under test. Before trusting a suite, confirm the thing you care about
appears in its output by name — 99 passing where 108 should pass is a skip,
not a pass.
Below the rules is an index of 29 skills — the ledger, deployments, QA flows, production release, the compliance register. A skill is loaded when the task needs it, never by default, which is how the always-on file stays small. This is the versioned-knowledge layer Part 2 describes being written by the learning loops; here it's just the filesystem it lives in.
Prose doesn't hold. Enforcement does.
Every rule in that file has been ignored at least once. Agents are not malicious; they're busy. So the rules that matter are enforced in three places, and the file is only the explanation.
1. Tool-layer hooks. A PreToolUse hook runs before every shell command the agent executes. This one denies the raw merge command and points at the wrapper that also tears the worktree down:
# bin/guard-gh-merge.sh — exit 2 denies the tool call; stderr goes back to the model
if printf '%s' "${command}" | grep -Eq 'gh[[:space:]]+(…)pr[[:space:]]+merge'; then
cat >&2 <<'EOF'
Blocked: use `mise run merge` instead of `gh pr merge`.
It squashes the same PR, waits for GitHub to confirm MERGED, sweeps other merged
worktrees, then closes this one — which is what frees the ~1Gi the branch holds.
EOF
exit 2
fi
The comment at the top of that script is the whole philosophy: nine of the worktrees merged on 2026-08-13 outlived their PRs, and a rule that is only written down cannot tell you which of them ignored it.
2. Tasks that refuse. mise run merge is the only door, and it says no on a draft, on untriaged Bugbot findings, or while stacked layers still base on the branch. mise run watch-checks refuses to wait for a check that can no longer arrive. A refusal with a reason is a teaching moment; a silent hang is four lost hours.
3. CI that fails on the boring things. Formatting is a failing check — markdown and JSON included. Lint is type-aware. gitleaks runs on every push. A workflow-lint job checks the workflows themselves. None of this is clever; all of it removes a category of review.
Nine gates on main
Here is what every PR into main has to survive, straight from the branch protection:
main- 1lintformatting + type-aware lint, markdown and JSON included
- 2unit-tests2,742 hermetic tests, after five structural invariant checks
- 3integration-teststhe Postgres-marked slice against a service container
- 4translation-catalogsno hardcoded user-facing strings
- 5workflow-lintthe CI workflows themselves
- 6gitleaksno secret in any commit
- 7Cursor BugbotAI code review on every PRevidence
- 8bugbot-triagered while any High finding is unansweredgate
- 9qa-gatered until the QA agent's verdict artifact says passgate
Two things to notice. The unit-tests job also runs five repo-specific invariant checks before the suite — Restate workflow shape, impersonation guards, date handling, source encoding, verifier bundles — because a suite that only tests behaviour misses the mistakes that are structural. And both AI reviewers come as a pair: the review (Bugbot, the QA drive) produces evidence; a separate, trusted job (bugbot-triage, qa-gate) reads that evidence and holds the gate. Untrusted code never certifies itself.
The suite behind gate two, today: 2,742 passing unit tests, hermetic by rule — no Postgres, no Restate, no mailpit — so they run in twelve seconds on a laptop and in CI without a service container. Tests that genuinely need a database carry a marker that routes them to the integration job instead.
The QA agent, and what it costs
Gate nine is the one people ask about. Since 13 August it runs in CI rather than on my laptop:
- A context job decides whether this deploy deserves a drive: an open, non-draft PR into main that touches a product surface and carries no waiver. Anything else publishes a legible skipped verdict on the PR instead of silence.
- The qa job is the untrusted half. It runs code from the PR head — the app, in a real Chromium — so it has no write permission and no secret beyond an API key. Its only output is an artifact.
- A publish job is the trusted half. It runs no PR code, reads the artifact, and writes the check run, the PR comment and the Linear mirror.
The agent's brief is read from main, never from the PR — a PR cannot rewrite the instructions of the process judging it. That's a Part 4 rule, enforced here in a workflow file.
Thirty days of that key's bill, by day:
QA Agent API key, by day, from the Claude Console. The three tall bars are the days it drove every push; the plateau is after it was scoped. Almost all of it is Sonnet.The shape is the tuning history. The first three days it drove everything — every push to every branch — and cost over $100 a day. Then it was scoped to PRs into main that touch product surfaces, with integration branches driven once per merge rather than per stacked PR. The plateau since is $40–80 on a busy day and single digits on a quiet one, almost entirely Sonnet. Across the month: 273 QA reports published, about $716 — $2.62 a report.
Worktrees and environments
Parallel agents need isolated workspaces or they trample each other and you'll blame the agents. Every task starts with wt switch --create <branch>; the worktree hooks run a two-second doctor, a five-minute prepare, and the migrations, then tear the branch's instance down on remove. A stale-worktree reaper runs hourly because a branch instance that outlives its PR holds about a gigabyte.
Any branch that needs QA gets a staging environment on demand, named after the branch, destroyed on merge. Agents do whatever they like there; production is reached only by a tag. The security model is Part 4; the plumbing is a deploy-staging workflow and a destroy-staging workflow, and nothing in between.
The ten-second test
Never claim a process you can't screen-share in ten seconds. Before adding anything else, open these five things with your tech lead:
Show me your last merged PR.
The checks ran on that change. A failing check would have stopped the merge.
See the detail →Open its ticket.
The brief, acceptance criteria and delivery status are there. The agent can read them without you relaying Slack messages.
See the detail →Open a current branch environment.
Its own URL, its own seeded database, synthetic data. Show the core flow reaching a completed state.
See the detail →Show me what the agent can access.
Name the environments, data and credentials. Show the permission boundary; an instruction to be careful isn't one.
See the detail →Show me the last paging drill.
A real check failed and the on-call phone rang. You know how long it took.
See the detail →
Can't show one? Start there. Can show all five? Measure how much of the process still waits for you.
Build order
If you're starting from a model and a licence, this is the order I'd build it in. Keep existing reviews and access controls in place while you do it. Earn more autonomy with evidence.
Write the contract alongside the first steps: a short AGENTS.md, the setup instructions, and a merge task that refuses when the gates aren't green and cleans up when the work is done. Enforce the important rules in the tools, as above.
Worktrees, properly.
One per task, an isolated database where the app needs one, and a setup script in the repo. Every agent starts here.
Done when: Start two tasks without either touching the other's files or data.
Make CI hold the line.
Require lint, types and unit tests on PRs to main. Add Bugbot or another reviewer, with a triage gate that blocks unanswered findings.
Done when: Deliberately fail a check on a test PR. The merge must be blocked.
Put the work back in tickets.
Linear or your existing tracker is the source of truth. Agents read it and write it. Have an agent draft the next ticket from approved, sanitised notes; check the acceptance criteria before it builds.
Done when: An agent can take a ticket through to a verified result without you copying messages between tools.
Give each PR somewhere to run.
A branch URL and its own seeded database. Synthetic data only. Tear the environment down when the work is done.
Done when: A teammate can open the change and try it without setting up a laptop.
Test the core flow in CI.
Run Playwright against the branch environment. Drive the important journey to completion and make the result a required check.
Done when: Break the journey on a test PR. Watch the test fail and hold the merge.
Make an agent use the product.
Add a browser-driving QA agent after the deterministic tests. Keep its instructions and the job that publishes its verdict outside the control of the PR being judged.
Done when: A failed or missing required QA verdict blocks the merge; a comment alone doesn't.
Finish the monitoring so the pager rings.
Agree a drill, trigger a real failure in a safe environment and exercise the full paging route. Time it. A staging-only Slack alert doesn't prove the production route.
Done when: The right person's phone rings, they acknowledge it, and you record the elapsed time.
The QA agent's code and README are a starting point for step six. Scope it from day one: PRs into main, product surfaces only. Count drives per day so a noisy trigger doesn't quietly become your biggest bill.
For the wider baseline — throughput, human minutes, quality and cost — use the six measures in Part 1.
The bill
The question the CTO actually asked was "thousands, or tens of thousands a month?" Neither.
Not in that table: GitHub Actions minutes, the AWS bill for branch environments, and my attention — briefing, judging designs, reading what the gates say. Part 1 makes the case that attention is the budget line that binds. It still is. But the money line is about the price of one engineer's afternoon, per month, for a process that merges nearly 700 PRs.
That's the harness. Now read what runs on it.
Audit your harness.
The first technical session I run with a CTO is a harness audit: what your agents read, what stops them, what a PR has to survive. Most orgs discover they have a model and no harness.
Email Ben about a reviewWorked with me before? Reply to my email and tell me where you got stuck. You can also find my contact details here.