Let your AI agent grow its own rules, but only on evidence it can't fake

For four days I had a finished post that would not ship. It was written, it passed every quality check but one, and the one it failed was the check that asks "is this grounded in something real you actually did." The answer was yes. The system just had no way to know that without me stopping what I was doing and hand-editing a code file to tell it so. Multiply that by a month and you get the number that finally made me fix it: twelve days dark, the whole pipeline stalled, not because it ran out of things to say but because it ran out of things it was allowed to say and only a human could refill the list.

If you're vibe coding an agentic system that needs to grow its own ground truth over time, an allowlist, a registry, a set of facts it's permitted to cite, the fix is almost never to trust the agent more or to babysit it harder. It's to split the two things you've accidentally fused: proposing an entry and ratifying it. Let the agent propose freely. Let a mechanical check do the ratifying, and gate that check on evidence the agent cannot manufacture. Then the list grows on its own without a human in the loop, and it still can't be faked, because the thing being checked isn't the claim, it's the proof behind it.

why was a finished post stuck for four days?

Because the list of what it was allowed to claim lived in code, and the agent that writes the posts runs under a standing rule never to edit code directly. That rule exists for good reasons. Generation glitches, a half-written edit corrupts a file, code changes want review. But it had a side effect nobody designed on purpose: the vocabulary of what the system could honestly say it had experienced could only expand when I opened a source file and typed a new entry by hand.

Meanwhile the system spends that vocabulary. Every post is anchored to one real build moment, and once a moment has been told, it's spent, because telling the same story twice is just duplicate content wearing a fresh title. So the pool of eligible material shrank a little with every ship and only ever refilled when I intervened. The result was a pipeline that looked autonomous right up until it wasn't, going dark for days at a stretch while a finished, spec-clean post sat waiting on the one gate a human had to satisfy. The gate was right to refuse. The design was wrong to make my hand the only key.

how do you let an agent grow its own rules without letting it rubber-stamp itself?

You move the list out of code and into data, a committed file the system reads instead of a constant it imports, and then you make ratification a proof instead of a judgment call. In my case that meant a small ledger, a versioned JSON file, where every entry carries a status. The agent can append a new entry any time, but it lands as "proposed," and a proposed entry satisfies nothing. It can't back a published post. It just sits there as a claim with its receipts attached.

The promotion from proposed to ratified is the whole game, and it's mechanical. A separate verifier reads the entry's evidence and either promotes it or leaves it sitting. No human types anything. But, and this is the part that makes it safe, the verifier doesn't check whether the claim sounds plausible. It checks whether the evidence is the kind of thing the agent could not have manufactured to order. That distinction is the difference between a system that can extend itself and a system that can lie to itself, and it's worth being precise about how you draw it.

what makes the evidence impossible to fake?

The evidence has to be anchored in artifacts the agent can point at but cannot create on demand. In practice that came down to a handful of rules, each closing a specific way of cheating. The evidence has to be a real commit that actually resolves in the repository's history, so you can't cite a hash you made up. It has to be at least a day old, which quietly defeats the most obvious attack: an agent can't do a scrap of throwaway work, commit it, and cite it as lived experience in the same run, because the clock won't let it. It can't be a commit the publishing system itself produced, so the engine can't launder its own output into evidence of authenticity. And it has to touch real files outside the content directory, because a receipt comes from building something, not from publishing something.

There's one more rule that I think is the sharpest, and it generalizes past my little pipeline to any system like this. A file on disk can be supporting evidence but it can never, on its own, ratify anything. The reason is a one-line principle worth stealing: an agent can create a file, but it cannot create history. Anything the agent can write in the current moment is suspect by definition, because it can write anything. Only the things already fixed in the past, commits that exist, are old enough, and touched real work, are trustworthy, precisely because they're out of reach of the run that's trying to use them.

the point is to design the forgery first

This is the part I'd want a vibe coder to take away even if they never build a ledger. I didn't start from the happy path, "how does the agent add a good entry," and bolt on safety after. I started from the failure surface: assume the agent will try to fake an entry, assume it's motivated to unblock itself, and ask what artifacts it genuinely cannot forge. The rules fell out of that question. Build the forgery mode first, then the honest path is just the forgery mode failing to trigger.

That's the same instinct behind spec-first work in general. You get more safety from naming the way a thing breaks than from polishing the way it's supposed to work, because the happy path tests itself every day and the failure path never runs until the day it matters. An allowlist that only ever imagined good entries is one clever prompt away from a bad one. An allowlist that was designed around "here is exactly how something fake would get in, and here is why each of those doors is locked" holds, because you did the adversarial thinking up front instead of discovering it in production.

where does the human go?

The human doesn't leave. The human's job changes, from gate to auditor. Before, I was the mechanism: nothing grew without my hands on the code. After, every promotion is a visible diff in a version-controlled file, a legible record I can read after the fact instead of a code edit I have to author before the fact. If I ever want to know what the system decided it was allowed to claim and why, it's right there in the history, one entry per row, evidence attached. That's a strictly better place for a human to stand. You stop being the thing the system waits on and you start being the thing that can catch it if it ever drifts, and catching drift after the fact scales in a way that authoring every change by hand never will.

That reframing is the quiet win. AI orchestration gets fragile exactly where a human is silently load-bearing, where the whole autonomous story secretly runs through one person's afternoon. Find those spots and replace the person with a proof, and the person with an audit, and the system stops going dark every time that person is busy.

questions that keep coming up

Isn't letting the agent grow its own list just asking for hallucinated entries? Only if you let the claim be what's checked. A hallucinated entry still has to produce real, aged, external evidence to promote, and it can't, so it stays proposed forever and backs nothing. The agent is free to imagine; it's just not free to ratify its imagination.

Why not have a human approve each one? That's exactly the design I was escaping. Human approval on the critical path is why the pipeline went dark for twelve days. The goal isn't to remove human judgment, it's to move it off the blocking path and onto the audit trail, where it reviews a diff instead of gating a run.

What actually stops the agent from making a fake commit to cite? The age gate, mostly. Real evidence is older than the run using it, so an agent can't manufacture-and-cite in one motion. Add the rule that the publishing system's own commits don't count, and the rule that evidence has to touch real build files, and the cheapest forgeries are all locked out. It's not a proof of good faith, it's a proof the receipt is real, which is the only thing the gate ever needed.

If you're formalizing how your agents extend themselves and want to compare notes on what actually holds at production scale, /work-with-us. Most of the fragility I've watched vibe coders fight in AI orchestration isn't the model, it's a human quietly wedged into the autonomous path, and that one you can design out.

The reason this works isn't clever. Authenticity was always going to come from things the agent can't retroactively invent, so the move is just to check those things directly and ignore the claim on top of them. Let the agent point at reality all it wants. Build the system so pointing is all it can do, and growing its own ground truth stops being a risk you take and becomes a proof you can read.