Blog · Part 2 of 3: Agents run your SDLC
Your agentic org self-improves
In Part 1 I showed you the six-step SDLC with agents doing every role. Here's the problem nobody warns you about: an agentic org has no institutional memory.
When an engineer hits a nasty gotcha — the migration tool that generates broken SQL, the CLI pin that can't talk to any deployed cluster — they remember it. They mention it at standup. It becomes tribal knowledge. When an agent hits the same gotcha, it burns twenty minutes solving it brilliantly… and then the session ends, and the knowledge dies in terminal scrollback. The next agent burns the same twenty minutes next week.
Human orgs fix this with retros. So yesterday I asked: what does a retro look like when the workforce is agents? By end of day we had one running. It backfilled two weeks of history, filed its first reports — and found five bugs in itself, four of them the same failure class. I'll show you the whole loop, including the embarrassing parts.
The principle: collect cheap, decide together
Everything below runs on one split I keep coming back to: collection is cheap and asynchronous; processing and implementation are collective and thought out.
Capture must cost nothing, or it stops happening — no ceremony, no meeting, no form. Judgment is expensive, so it runs on a cadence, in batches, where patterns are visible. And decisions about what to build stay human, because that's the part where taste and strategy live.
The corollary that keeps it safe: no LLM in these loops runs in CI, and no judgment runs on a cron. Scheduled jobs are deterministic — count things, compare dates, file tickets. All the thinking happens in agents that a ticket hands work to, and all the deciding happens with me.
A hook reflects on the session transcript. One tagged what / why / evidence entry — or NO-ENTRY.
5 pending entries, or one 14 days old, files a consolidation request. Clusters become small doc PRs.
One report: metrics row, friction by tag, ticket ages — and an audit of the last retro's decisions.
The pruning pass: stale rules, tickets nobody actioned, lessons that stopped recurring.
Loop 0: the maintenance detectors (already running)
Before the retro existed, we had the base layer: ~17 deterministic detectors on a daily heartbeat. Each one checks something boring — stale staging environments, docs drifting from code, dependency advisories, unpinned CI actions, ISO evidence going stale — and files findings as one Linear ticket per detector, updated in place. Labelled needs-cursor, an agent picks it up; the heartbeat itself never thinks.
The design lesson came from the first one. env-cleanup politely filed the same ticket every day; nobody acted; staging ran out of capacity to schedule anything. Reporting alone doesn't work — so that one detector earned the right to act (it dispatches the teardown pipeline, which has its own guards). Every other detector still just files tickets. A detector earns teeth by demonstrating that the ticket alone failed.
Loop 1: capture — every merge teaches the system
Now the new part. The moment a session runs the merge command, a hook spawns a background reflection over the session transcript: did anything in here cost time that a doc, skill, or process change would have prevented?
Its entire output is either one entry — what happened, why it cost time, evidence, proposed fix, plus one or two tags from a fixed vocabulary — or the literal string NO-ENTRY. A tiny script posts it as a comment on a standing Linear ticket. The model never touches Linear; the script never judges.
We backfilled it over the last two weeks of real sessions:
ENTRY 4e4ddce3 NO-ENTRY 0bc71d72 NO-ENTRY 5746fd9d
ENTRY 449ba2fa NO-ENTRY 2dca2efe NO-ENTRY bdd009f5
ENTRY faf5c17e NO-ENTRY ee1779b9 NO-ENTRY 9219e621
ENTRY 1f173552 NO-ENTRY 22edcb9b NO-ENTRY 4c760709
backfill complete: 12 transcripts considered
Four lessons, eight NO-ENTRYs — plus a fifth the system had already captured about itself while we tested the capture prompt. The NO-ENTRYs are the feature. Most sessions have nothing durable to teach, and a reflection that admits it is what keeps the log worth reading. The five that survived were all real: a .gitignore append that silently fused two lines because the file lacked a trailing newline; a database migration tool whose generated enum migrations fail outright in Postgres; a CI run that a GitHub outage left simultaneously "already completed" and "already running".
Loop 2: the batch gate — nothing acts on one entry
A weekly detector counts pending entries. Five entries — or one that's been sitting two weeks — files a consolidation request and hands it to an agent: cluster the entries, dedupe against existing docs, and turn recurring lessons into small doc PRs against the repo's agent guidance. Normal review, normal merge.
Why batch? Because "three sessions hit the same friction" is a signal no single session can produce. Acting on individual entries is how you get doc churn nobody reads. The noise bar lives at the batch, not at capture.
And this is the part that compounds: in Part 1 I said we keep ~30 versioned skill files in the repo instead of vendor "memories". This loop is what writes them. Lessons from agent sessions become docs that every future agent session reads — the org's knowledge base maintains itself, one reviewed PR at a time.
Loop 3: the sprint retro — deciding what to build
Every two weeks, one report lands on a standing ticket. It opens with a metrics row — merged PRs, entries captured, consolidations, open automation tickets, each windowed since the last report, so the ticket's comment stream is the time series. No dashboard to build, no dashboard to forget.
Then the sections: high-friction points (log entries clustered by tag, plus every open automation ticket with its age — our first report showed six tickets sitting unprioritised for a month, which is the retro making its own case), routine upkeep like version bumps, and — the part I stole from reading twenty years of retro literature in an afternoon — the audit comes first: the report quotes the previous retro's decisions and asks what actually shipped.
A prep agent digests all of it into a ranked one-pager: at most three friction clusters, at most two candidate investments, and a papercut list where every item is under a day of agent work. Then a human decides, in a comment, which the next report quotes back. The research on why retros die is unanimous — long unowned action lists and rituals nobody reviews. The two-investment cap and the audit-first rule are the guards. Every sixth retro adds spring cleaning: prune stale rules, close ignored tickets, delete docs for lessons that stopped recurring. Rule files are trained weights; nobody schedules the pruning by default.
It debugged itself before lunch
Here's the honest bit, and the part I'd lead with if I were selling you nothing.
Day one, the system caught five real defects in itself — three via the mandatory Bugbot gate from Part 1, two via sceptical reads of its own output. Four of the five were the same failure class: a text system matching text it wrote itself. The routing label attached before any batch was due. The decisions-audit that would have quoted a report back as a decision, forever. And my favourite: the consolidation-request comment mentions the consolidation marker in its instructions — so substring matching counted the request as the consolidation, and the batch reset itself the moment it was declared due. The first metrics row proudly recorded a consolidation that never happened.
Each fix shipped the same day with a regression test, through the same gates as any other PR. And the first entry the system ever logged — parse structured output by structure, never by position, because harness noise leaks into everything — is the exact lesson those fixes encode. The learning system's first lesson was about itself.
That's also your answer to "how do I trust this?" — the same answer as Part 1. You don't trust it; you gate it, measure it, and read its first outputs with maximum suspicion. The metric that keeps everyone honest long-term: does a lesson recur after its doc fix merged? A recurrence means the fix didn't work. That's the difference between a learning system and a diary.
Round two: adversarial agents vs. the learning system
We didn't stop at shipping it. The same afternoon we ran the repo's multi-agent review pattern against the pipeline itself: five finder agents, each with one adversarial lens — Goodhart gaming, silent-death paths, prompt signal quality, human factors, lean-ness — working blind to each other, each capped at three evidence-backed findings. Then frontier-model refuters, each briefed to kill a finding by reading the actual code.
Nineteen agents, six minutes, ~1.6M tokens. Fifteen findings came back; the refuters killed three of them — one by empirically probing the live ticket API, another by opening all fourteen of the day's capture transcripts and checking the claim against reality. That 20% rejection rate is the whole argument for the two-stage shape: without the refuters, three plausible-sounding non-bugs would now be "improvements" in my backlog.
The theme of what survived: the pipeline measured its own activity but not its own health. A dead capture hook rendered identically to a healthy quiet fortnight. Nothing actually notified a human that a retro was waiting. A consolidation could be marked done by an agent that dropped every entry — indistinguishable, to every downstream signal, from one that shipped real doc PRs. All fixed the same day: the metrics row now flags zero-entry fortnights, the retro ticket assigns itself to a human, and a consolidation close gets audited — the detector parses the disposition lines and resolves every claimed PR against GitHub before believing the batch happened.
Two details I'd frame on the wall. The review bot caught that my "assign the ticket" fix had the exact same bug class it had flagged on day one — the reviewer gate now recognises this system's signature failure mode faster than I do. And the seven verified-but-minor findings weren't dropped: they were posted to the retro ticket as its first papercut candidates. The adversarial sweep's leftovers became the retro's input. The loop closed on itself.
Steal this
The whole thing is four small pieces, none clever: a merge hook that reflects on the transcript; a script that posts entries; two deterministic detectors with self-owned cadences; two prompt files that carry the judgment. It cost one day across five PRs, most of which was the system finding its own bugs — which, again, is the point.
If your agents ship fast but every session starts from zero, you don't have an engineering org. You have a very expensive Groundhog Day.
Part 3: Your agents react — incidents, outages, and production access.
I coach VPs of Engineering and CTOs through this shift.
If your agents ship fast but never get smarter — that's a Groundhog Day problem, and it's fixable.
Talk to me