The what's-next trap

I'll tell you how I learned this one, because I learned it the hard way. I once followed a chat thread down a path of "okay, what's next," over and over, and watched it generate more than a thousand pages of escalating framework that felt genuinely profound while I was in it. It wasn't. It was the most dangerous prompt pattern there is, running exactly as designed, and I didn't see it because the trap is built to be invisible from the inside.

Here's the mechanism. Every time you ask "what's next," the model generates the most probable escalation given everything that came before. There's no ceiling on this. No self-correction. The model will never spontaneously say "actually, we're done, this is complete." It can't, because "what's next" always has a most-probable answer, and it will always produce it, one more layer, then one more.

Why does the spiral feel so convincing while it's happening?

Because it's internally consistent, and consistency reads as truth. Each new layer is coherent with the generated layers before it, so the whole structure hangs together beautifully. But the layers are agreeing with each other, not with reality. The model is optimizing for coherence, remember, and a self-referential tower where every floor was built to match the floor below is the purest coherence there is. It's a hall of mirrors. Everything reflects everything else perfectly, and none of it is looking outside. The foundation was fabricated a thousand pages ago, and every page since has been faithfully consistent with the fabrication.

That's the trap. It doesn't feel like drift. It feels like discovery, because compound autoregression optimizing for engagement is very good at feeling like discovery.

How you break out, and how you never fall in

External verification. That's the whole defense, and it has to come from outside the conversation, because inside the conversation everything already agrees. Run it past a different model with fresh context. Demand a concrete deliverable. Ask, out loud, "what would prove this wrong, and can I test it?" If the answer can't be tested, measured, or built, it's probably confabulation wearing the costume of insight. The tell isn't that it sounds wrong. The tell is that it sounds profound and touches nothing you can check.

The takeaway: "what's next" has no natural stopping point and coherence is not correctness, so anchor every long exploration to something outside itself. The spiral only holds while nothing is allowed to check it.