The private line my chatbot could hand a stranger

A few days ago I sat down to attack my own chatbot. Not to check whether it answered questions well, but to see what it would give up if someone came at it sideways. I typed the kind of thing a nosy visitor types, some variation of are you the person behind this, and the bot answered by quoting a line straight out of my private notes. The line had my real name in it.

Nothing had failed. Every check was green. The bot was doing exactly what I built it to do, which was retrieve the most relevant lines from my own material and answer in context. The problem was that one of those lines was a fact about me I never meant to hand to a stranger, and the question I asked was the exact key that unlocked it.

Can a chatbot leak private information even when your code is clean?

Yes, and this is the part worth internalizing if you are building anything on top of your own material: your retrieval corpus is a publication surface. Every file your bot can pull from at answer time is effectively public, because a determined visitor can phrase a question that surfaces it. The fix is not to write more carefully. It is to point your secret and identity scanning at the content the bot retrieves, not only at the code you wrote, and then to prove the corpus is clean with a test that drives the real retrieval path instead of trusting a static read.

That distinction is the whole lesson. Most of us who do vibe coding think about secrets the way traditional security tooling teaches us to: scan the source, block the API key in the commit, keep credentials out of the repo. That instinct is correct and it is also aimed at the wrong half of the system when the app is a chatbot. A retrieval bot does not just run your code. It reads your content and speaks it. The moment you point a model at a folder of your own notes, that folder becomes part of the app's mouth.

Why the guard I already had could not see it

Here is the embarrassing shape of it. I had a gate for exactly this. A pre-commit hook ran an identity scan on every commit, looking for names and other protected values, and it had been passing clean for weeks. It scanned the application code, the components, the shared library, and the public assets. It did not scan the content directory.

The one tracked file that held my name was in that content directory. It was, of all things, the private rule that told the bot never to disclose who I am. To make the rule concrete, the file quoted my name as the example of what not to say. So the guard built to catch that string could not see the single file that contained it, and the file that leaked was the file about not leaking.

Retrieval made it worse in a way that feels almost designed to trip you. Snippet extraction centres on the query terms. When a visitor asks for the protected value directly, the retriever does its job perfectly and pulls the passage most about that value, which is the line holding it. The better the retrieval, the more reliable the leak. A vibe coded app that retrieves well is not safer here. It is more exposed.

The move: put the gate on the surface where the failure lives

The principle I keep coming back to is failure-surface-first. A gate is only as good as the surface it watches, and a gate that watches the wrong surface is worse than no gate at all, because it pays you in confidence you did not earn. Green checks stop feeling like evidence and start feeling like a promise, and you ship on the promise.

So the fix was less about the one line and more about moving the check to where the exposure actually happens. Four changes, in order of how much they mattered:

First, I extended the scan to cover the content the bot retrieves, not just the code. That is the one change that would have caught the original leak. The denylist moved into its own pattern file that lives outside the scanned surface, so the list of protected values is never itself a thing the scanner trips over or a thing the bot can read.

Second, I made the check fail closed. If the pattern file is missing or empty, the scan does not quietly pass with nothing to look for. It blocks. A check that cannot run must never report clean, because a silent pass is how you end up trusting a gate that has been asleep for a month.

Third, I rewrote the private file to describe the shape of a protected question rather than quote an instance. The rule now says the bot must refuse questions of the form who are you, where are you, who employs you, without ever naming the answer. You cannot leak a value you never wrote down.

Fourth, and this is the one I would not skip again, I added a regression test that drives the real retrieval path. It builds adversarial queries out of the denylist itself, runs them through the actual retriever the way a hostile visitor would, and asserts that nothing protected comes back in either the model context or the public snippets. It hardcodes no protected string of its own. The negative control matters: on the old file the test fails on both the static string and the live retrieval, and on the fixed file the whole suite passes. If a test does not fail on the broken version, it is decoration.

What this means if you are shipping a bot over your own material

If you are a vibe coder who has pointed a model at your own notes, docs, support tickets, or an about page, treat every one of those files as if it will be read out loud to the least friendly visitor you can imagine, because it can be. Run your secret and identity scanning against that content, not just your source. Assume that good retrieval will find your worst line faster than a human ever would. And write a test that asks the hostile questions on purpose, because the only honest proof that your bot will not say a thing is watching it try and fail.

The reason this class of bug slips past careful people is that it sits in the seam between two mental models. Content feels like writing, so we edit it. Code feels like machinery, so we scan it. A retrieval app fuses the two: your writing becomes machinery the moment the model can quote it. The gate has to live on that fused surface or it is guarding a door that is not the one anyone walks through.

If you have already shipped something that answers from your own private material, it is worth an hour to sit down and attack it the way I did, asking the pointed questions and watching what it retrieves. If you want a second set of eyes on your retrieval surface, auditing what a hostile question can pull out of a vibe coded app is one of the things I do, and you can start that conversation at /work-with-us.

Questions that keep coming up

Does this only matter if my notes contain secrets like passwords? No. The riskier leaks are usually not credentials, which good tooling already catches, but ordinary personal facts, names, locations, client identities, internal reasoning, that felt safe as private writing and become exposure the moment the bot can retrieve and speak them.

Will a system prompt telling the bot not to reveal something protect me? Not reliably. A standing instruction helps, but the value still sits in retrievable context, and a well-phrased question routes around the instruction by surfacing the source line directly. The durable fix is to keep the value out of the retrievable corpus, not to ask the model nicely to forget it is there.

Is this the same as prompt injection? It is a cousin. Prompt injection is a visitor smuggling instructions in. This is a visitor pulling your own data out with a normal-looking question. The defense overlaps, but the root cause here is that private content was reachable at answer time, which is a data-boundary problem before it is a prompt problem.