Agent Engineering
I wrote the rules my agents work under
Course Kit shipped thirteen releases in eight months across three platforms, built by one person. This is the system underneath that: a written doctrine, a routing ladder with a ceiling, and a set of hooks the model does not get a vote on.
The problem
One person cannot review one person's work.
Agents made me faster in the first week and unreliable in the third. A subagent reported a change complete and gave me a file and a line number. The line number did not exist. It was not lying. It had no way to check, and neither did I, because I was the one who had asked for it.
The bottleneck was never generation. It was trust.
Rules restated in every prompt · Agents grading their own work · Retries with no ceiling.
Rules written once, in a file · Blind verification by a fresh agent · Five attempts, then a person.
What I built
A doctrine, not a prompt library. Six documents that answer the questions I kept re-deciding: who does this work, on which model, how do we know it is finished, and when do we stop and ask a human. Written for a model to read, with a worked right answer and a worked wrong answer under every rule.
Around it sit eighteen skills in regular use, five shell hooks that enforce what a prompt can only request, and an operational memory corpus of single-fact documents with an index, an archive tier, and a dedup rule on write.
Where it runs
The doctrine was written to ship code. It now carries the rest of the company: strategy, marketing, writing and operations each have their own agent roles, and all of them work under the same dispatch, verification and escalation rules. A landing page draft is accepted the way a schema change is: against written criteria, by an agent that did not produce it.
Decisions
The writer never grades the work
Verification goes to a fresh agent that gets the acceptance criteria and the artifact path, and nothing else. A verifier told how the work was done will agree with it. File claims resolve to a read-back, code claims to the project's own check command with its raw output attached.
Escalation with a ceiling
Mechanical work to the cheap model, implementation to the middle tier, adjudication to the strongest. One retry at the bottom, two in the middle, two at the top, then it stops and hands back to me. Escalation without a ceiling is an expensive infinite loop.
Prompts request, hooks enforce
Destructive git commands, banned punctuation in user-facing copy, output paths that must not be written to. None of those belong in a prompt. They live in shell hooks that return an error the agent can act on, and a hard stop for anything irreversible.
Taste is routed to a person by policy
Decomposition and review raise execution quality. They do nothing for "is this copy any good". Those questions come back to me as two or three concrete options with their costs. The model is not permitted to sound confident about a question with no verifiable answer.
The interesting engineering
Three payment rails taught me the pattern. Agents made me apply it to myself.
Course Kit normalises Apple, Google and Stripe into one entitlement schema, and a conflict resolves to the highest tier rather than the last write. Agent output has the same shape of problem: several sources, different confidence, and a wrong default. So the same rule applies. A claim resolves to the strongest evidence, not to the most recent speaker.
In practice a file claim resolves to a read-back, a code claim resolves to the project's own check command with its raw output attached, and anything touching schema, payments or permissions resolves to a second independent opinion before it ships.
Results
The first four numbers are the point. The rest is what made them survivable: an operational memory corpus in the largest project, and a hard limit on how long any agent is allowed to keep trying before the work comes back to me.
A rule I had to correct
I wrote this rule, shipped it, and then measured what it actually cost. It was wrong in a particular direction: it optimised for looking disciplined rather than for the thing it was supposed to defend.
| Rule | What I shipped first | What it cost | What I changed |
|---|---|---|---|
| Delegation threshold | Any read over 300 lines goes to a subagent, to keep the main context clean. | Ritual delegation Eighty-line reads were being dispatched at roughly three to four times the token cost of reading them directly. | Kept the threshold, wrote the exception into the rule, and named the real target: a localisation directory of 35,000 lines that costs 40k tokens to read whole. A guideline that cannot be violated is a bug. |
What this is not
There is no model in front of my users. Course Kit ships payment rails and sync, not inference. Everything above is production-side: the system that builds and ships the product, and the rules that make its output safe to trust.
If you are hiring for model-facing product work, that is the gap. I would rather name it than have you find it.