Hallucination is the default, not a bug

The single most expensive misunderstanding about these models is treating hallucination as a malfunction, something that happens when the model breaks. It isn't. It's the normal mode of operation, and once you accept that, you stop being surprised and start defending against it.

Here's the uncomfortable core. The model optimizes for coherence, not truth. It's built to produce text that fits, that reads as plausible given everything before it. Whether that text corresponds to reality is a separate question the machinery never actually asks. When the true answer and the coherent answer line up, you get a correct response. When they don't, you get a confident, well-formed, completely fabricated one, and nothing in the model's behavior flags the difference.

Why doesn't the model just say when it's unsure?

Because it has no uncertainty signal. This is the part that should change how you work. The model does not know what it doesn't know. It cannot tell the difference between recalling a fact and inventing one that sounds right, because both come out of the same next-token machinery at the same confidence. Hallucination correlates with distance from the training distribution: common, well-worn knowledge is low risk, and niche specifics, recent events, exact numbers, and URLs are high risk. But the model won't warn you when it crosses that line. It'll answer a question about a made-up API the same way it answers what two plus two is.

The way I hold it is a rug pull. The chart looks perfect, the answer looks clean, and there's no liquidity behind it. And the defense is the same as in trading: don't take one source's word on a real position. Multi-AI consensus, running the same question past Claude and ChatGPT and Grok, is checking the same trade on three different DEXes. When they agree, your confidence is earned. When they diverge, you just found the thing to verify.

What to do with this

Verify anything niche, numeric, recent, or load-bearing, and treat confidence as no evidence at all, because confidence is exactly what the model produces whether it's right or wrong. This is the mechanical reason "verify, don't trust" isn't a personality trait, it's a response to how the thing is built.

The takeaway: the model optimizes for coherence and has no uncertainty signal, so hallucination is the baseline and verification is not optional. Trust the process you built around it, never the confidence it hands you.