When one AI reviews another AI's code

A couple weeks ago I handed a change to a second agent and asked it to review the work before I merged. It read the diff, reasoned about the logic, and came back clean: no objections, a tidy summary of what the code did, a green light. I merged it. The bug I had been worried about was still sitting right there, in the part it told me looked fine.

If you are a vibe coder asking whether you can trust an AI to review AI code, here is the short version. You can use it, but a green review from a second model is a surface signal, not proof the change is right. A reviewer that reasons the way the author reasoned inherits the same blind spot, so it tends to bless exactly the mistakes you most need a second opinion on. The real second opinion is an independent check against the actual diff and the running app, plus a hard limit on how much you will wave through as a "small fix" without looking yourself.

This showed up again in the wild this week. Someone benchmarked eight AI code reviewers against sixty-seven real production bugs pulled from well-known open codebases, and the practitioner threads underneath were full of the quieter version of the same problem: an agent that "did something git-wise you didn't see coming," a tool that left the workspace a mess it never mentioned. The instinct to benchmark reviewers is the right instinct pointed at the wrong fix. Ranking which model catches the most bugs still assumes the reviewer's verdict is the thing you trust. It is the verdict itself that is the problem.

why does a second AI miss the same bug?

A code review measures a surface signal. A reviewer reads the diff, reasons about whether it looks correct, checks it against what it expects the code to do, and turns green when nothing looks wrong. That is real work, and it catches real things. It is also a smaller claim than the green check makes it sound. Green means a reader that reasons like the author found nothing that offended its expectations. Whether the change is actually right, against real inputs, in the running system, is a different question the review never had to answer.

Vibe coding sharpens this. When the code and its review come out of the same kind of model, prompted the same way, the review inherits the code's assumptions instead of testing them. If the author decided a missing value should quietly become a default, the reviewer reads that line, recognizes the pattern as normal, and nods. Both agree, in perfect green, because they are drawing from the same well. I keep coming back to one line for this: surface signals propose, semantic measurement disposes. A green review proposes that the change is fine. Only a semantic check, is this actual output the actual right output in the real environment, can dispose of the question, and that check has to come from outside the author's own head to mean anything.

the two bugs a green review walked past

I learned this the un-fun way on a build a while back. I had the change looked over, it read clean, and I still did not trust one corner of it. So I stopped asking for opinions about the code and set up an independent audit pass instead: a separate agent that never saw the original reasoning, handed one job, ignore whether it runs and tell me whether the answers are right, checked against the real outputs and the diff rather than my summary of what I meant.

It came back with two. The first was a value that defaulted on failure. When an input did not parse, the code fell back to a plausible-looking number and carried on, and the review had read that fallback as deliberate because, structurally, it looked deliberate. The second was staler: a path that served a cached value when a fresh fetch came back empty, so the screen showed last week's answer under this week's timestamp. Both were non-throwing. Both had been reviewed. Both were exactly the kind of thing a same-reasoning reviewer is built to miss, because nothing about them looks wrong from the inside. That is the whole lesson in one receipt. The review told me it ran and looked normal. The audit told me it was wrong.

how do you actually get a second opinion?

Treat the reviewer's verdict as a proposal, not a grade. A clean review is the start of checking, not the end of it.

Check against the artifact, not the summary. The thing you trust is the real diff and the running app: the actual number on the actual screen, the record that actually saved. A reviewer telling you what the code does is describing its own reading of the code. Go look at what the code did.

Gate on scope, hard. The single most useful habit I have picked up is distrusting the size of a change relative to how "small" it was sold as. A two-hundred-line diff described as a small fix is a red flag, not a convenience. Big quiet changes are where the off-screen surprises live: the git moves you did not ask for, the cleanup that never happened, the three files touched to patch one. Cap what you will accept unread by eye, and make the agent justify anything past the cap.

Get a reader from outside the author's reasoning. An independent audit pass, an agent told only to verify the outputs with no stake in defending the approach, catches the class of bug a same-lineage review is blind to by construction. It is the software version of not proofreading your own writing. You read what you meant. A stranger reads what is on the page.

where trusting the review bites hardest

The pattern is worst exactly where the review is most reassuring. A tidy green summary lowers your guard right before the part that needed it raised. And once you trust the verdict over the artifact, the failure compounds: you re-prompt against a bug you do not believe is there, the agent "fixes" things that were fine, and the blast radius grows with every confident pass. This is the same shape I hit when the tests passed but the data was fake, and it is why I now verify what the agent actually changed instead of trusting the report of it. A clean review, a green suite, a confident summary: all proposals about correctness, none of them proof.

Real AI orchestration is less about finding a smarter reviewer than about owning the definition of proof. The reviewer is a fast first reader. It does not get to be the thing that says the change is right.

If you are shipping a vibe coded app and there is a change you had reviewed but still do not fully trust, /work-with-us. Send me the diff and the outputs you keep re-checking by hand, and I can run an independent audit pass over the real behavior, the kind that assumes nothing and checks values instead of verdicts, and set up the scope gates that catch the big quiet change before it lands. Work with VibeKoded.

questions that keep coming up

Should I use an AI code reviewer at all? Yes, as a fast first reader. It will catch obvious slips quickly and cheaply. Just do not let its green light stand in for a check against the running system.

Does a different model as the reviewer fix it? It helps at the margins, and it beats reviewing with the exact same setup, but a second model prompted to reason about your code still reasons about your code. The blind spot shrinks; it does not vanish. The independent audit against real outputs is what actually moves the odds.

What do I check by hand? The outputs and the diff size. Confirm the real values are right in the real environment, and treat any change much bigger than advertised as something to read yourself before you believe any review of it.