Engineering · 8 min
Reviewing AI-written code you will own
A working engineer's checklist for generated code: where the models fail structurally, what review has to catch now, and how to keep the accountability where it was.
See the engineering practiceRead once
The economics of review inverted
For most of the history of this job, writing was the expensive step and review was the cheap one. That ratio is what code review was designed around: a diff arrived slowly, so a careful reader could keep up with it.
Generated code inverts the ratio. Producing a plausible thousand-line change is now nearly free, and reading it costs exactly what it always did. Teams that do not notice this ship the review queue as their bottleneck and then, predictably, start approving on vibes.
The response that works is not to review harder. It is to review differently: to spend your attention where models fail structurally rather than distributing it evenly across a diff that was cheap to produce.
Where generated code fails structurally
These are not random errors. They follow from what the model has and does not have: it has an enormous amount of code that looks like yours, and it does not have your system, your invariants, or your last outage.
- Concurrency and idempotency. Code that is correct when run once, and wrong when run twice or run in two places. Retries, webhooks, queue consumers, and anything with an at-least-once delivery guarantee.
- Error paths. The happy path is usually fine. The catch block swallows, the partial write is never rolled back, the failure is logged and then treated as success.
- Boundaries. Trust assumptions about input that hold in the example and not in your request handler. Authorization checked in the caller and assumed in the callee.
- Invariants it was never told. The rule that lives in one team's head, or in a migration from two years ago, and is nowhere in the file the model was shown.
- Plausible APIs. Method names and options that ought to exist. Type checking catches many of these and not the ones in dynamic configuration or a string.
- Tests that assert the implementation. They pass, they will keep passing after a real regression, and they are worse than no test because they buy confidence.
Review the change, not the text
Start from what the change is supposed to do and whether this is a reasonable way to do it. A diff that is well written and solves a problem nobody has is still a rejection, and it is a rejection generated code produces more often, because producing it was easy.
Then go straight to the parts that touch the world: writes, deletes, external calls, permission checks, anything that spends money or sends something. Read those closely, line by line, in the way you would read a colleague's first change to that subsystem. Skim the rest.
Ask the author what they checked and what they did not. Not as a gate, but because the answer is the actual signal. Someone who says they ran it against the staging replica and watched the duplicate-key case is telling you something. Someone who says it passed CI is telling you the tests passed.
Accountability does not move
The name on the commit is the name on the incident. This is worth saying plainly on a team, early, before the first time it matters, because the alternative develops on its own: a quiet shared understanding that a class of bug is the tool's fault and therefore nobody's.
That does not make the tools a bad idea. It makes the ownership rule explicit. If you merged it, you can explain it. If you cannot explain it, it is not ready, and the honest reason to hold the change is the one you should give.
The same rule is what protects the person using the tools well. An engineer who reads what they ship should not be assigned the same suspicion as one who does not, and the only way to distinguish them is a norm about what merging means.
What to put in the automation instead
Volume is the problem, so machines have to absorb the volume. The leverage is in the checks that cannot be bargained with.
- Types at the boundaries, and a linter for the patterns your team has been bitten by rather than a generic ruleset nobody reads.
- Property and fuzz tests for anything parsing, merging, or reconciling. Generated code is good at the case in the example and bad at the case in the wild.
- A test that actually fails when the behaviour changes. Delete a line of production code and confirm something goes red, before you trust a suite you did not write.
- Migration and rollback rehearsed, not assumed. The generated migration is confident about a table it has never seen at your row count.
- Observability on the new path before it carries traffic, so the first report is a graph rather than a customer.
Where Datum fits
This is a working practice, not a product, and most of it belongs in your own repository. Datum exists as a place to keep the judgment sharp between incidents: short situations where the code runs and the question is what you would do before merging, feedback on the call you made, and a record you own rather than a completion badge. Core is free.