The deploy that never fired

A while back I pushed a post to this blog, watched the run go green, closed the laptop, and went to do something else. The post did not exist. Not on a feature branch, not half-built, not broken. The commit was clean and it was sitting on main exactly where it belonged. The page it should have become just was not there, and it stayed not there for about five hours, serving a 404 to anyone who went looking, while every signal I had told me the work was done.

If you ship a vibe-coded site or app through a git-to-host integration, the thing worth internalizing is this: your push and your deploy are two different events, and the second one can fail without turning anything red. A clean push is not a live page. The only signal that actually means shipped is the URL your readers load coming back with a 200. So the fix that holds is not a better push step. It is a check, run after every push, against the real page in the real place, because that is the one measurement the green checkmark on your commit never makes for you.

a clean push, a dead page

Here is what happened. The pipeline that publishes this blog drafted the day's post, ran every gate, moved the file into place, committed it to main, and pushed. Green the whole way down, and this time the branch was even right. The commit was real, the file was well formed, the push landed on the branch the host builds from. Everything the pipeline could see said the job was finished.

But the host builds on a webhook. The push is supposed to poke a separate system that says "something changed, go build it," and that separate system turns the new commit into a live page a minute or two later. That morning the poke never arrived. The webhook dropped, the build never fired, and the page that would have existed simply did not. There was no error to read, no red check, no log line to grep, no alert to wire up. The commit and the missing page sat there quietly together until I went looking for the post and found a hole where it should have been. Recovery was one empty commit that poked the host by hand, and the page came up within two minutes. Cheap to fix. The expensive part was the five hours of not knowing.

why a green push isn't a live page

The whole appeal of vibe coding is that you move fast on green checks and trust the machinery underneath to carry the work the rest of the way. Almost always that trust is earned. The push works, the build fires, the page appears, and you never think about the seam between those steps because you never have to. Then one day the seam opens.

I wrote about a cousin of this failure in the commit that shipped to nowhere, where the pipeline committed to the wrong branch and the deploy watched a branch that never moved. That one was about where the commit landed. This one is quieter and, in a way, worse, because the commit landed exactly right. The push was perfect and the ship still did not happen. It is also not the same as the gap in works in preview, breaks on deploy, where the build ran but shipped the wrong files. Here the build never ran at all. Three different ways for a green terminal to produce a page nobody can read, and the common thread is that the terminal was reporting on the push, not on the page.

how do you verify a deploy actually went live?

The move that fixed it is boring and it holds. After the push, the pipeline polls the production URL of the exact thing it just shipped and waits for a 200, with a generous timeout that covers a normal rebuild. The route 404s until the new build includes it, so a 200 is the first moment the deploy is provably real. If the poll times out, the pipeline does exactly one thing: an empty commit that nudges the host to rebuild, then it polls one more time. If the page is up after the nudge, it logs that a nudge was needed and moves on. If it is still not there, it stops and shouts, writing a loud durable alert that points straight at the integration, and it does not retry in a loop.

Two disciplines matter more than the polling itself. The first is that the nudge is bounded: one poke, never a thrash, because a retry loop against a broken integration just buries the real problem under noise. The second is that every needed nudge gets logged. A webhook that drops once and recovers looks like nothing in the moment, but a webhook that drops every few days is a real integration fault, and you only see the pattern if each self-heal leaves a mark instead of vanishing.

the guard that caught the next one

The reason I trust this one is that it has already paid out. Two weeks after it went in, a post 404'd eight minutes after a clean push, the same dropped-webhook shape as the original. This time nothing waited on me. The check noticed the page was not there, fired its single nudge, and the deploy came up. A failure that used to cost five hours and a human happening to notice cost one automatic commit and zero attention. That gap, five hours of silent 404 versus one logged nudge, is the entire return on building the guard, and it is why the guard was worth more than any amount of being more careful about the push.

observability you enforce, not observability you hope for

The dangerous thing about a dropped webhook is that the happy path trains you out of checking. It works ninety-nine times, so you stop looking on the hundredth, which is precisely the one that fails. A dashboard does not save you here, because the whole failure is that nobody was looking, and a dashboard is a thing you have to remember to look at. The version that survives a tired operator and a 5am scheduled run is the one where the system carries the watching for you.

That is observability-as-invariant, and it comes down to a single habit: name the signal that actually means done, then make the machine confirm that signal on every ship, not a proxy for it. The checkmark on a commit means the code was accepted. It does not mean a reader can load the page, and a reader loading the page is the entire job. When the thing you check is the real outcome instead of a stand-in for it, a dropped webhook stops being a five-hour outage you discover by accident and becomes a one-line nudge the pipeline files on its own.

If you are a vibe coder wiring AI to build and ship real software on its own, this failure is already in your setup, armed, whether or not it has fired yet. The push will be clean, the run will be green, and one day the page will not be there, and you will learn it from the silence rather than from any alarm. If you are formalizing an AI orchestration that deploys to production without a human watching and want a sparring partner on where it can fail quietly, /work-with-us. The check that catches a ship nobody would have seen is exactly the kind of invariant worth speccing on purpose, before it bites instead of after.

The boring version of the lesson: trust the page, not the checkmark. The checkmark tells you the code was accepted. Only the page tells you a reader can see it.