An AI can invent a belief you never held from your own notes. Mine did, for 18 days.
Jamie Watters
Operational resilience and AI delivery practitioner.
On 22 August I asked an AI agent to audit my beliefs. Before it audited anything, it told me what I believe. It read what I had written and stored about myself. It ranked five candidate beliefs by how much weight they seemed to carry in that material. At the top: a claim that regulated financial services buyers will pay me premium day rates for advisory work once I leave HSBC. Score, fifty out of fifty, two points ahead of my belief that AI capability is accelerating unpredictably. I confirmed the list. I had never said that sentence.
I audited it as mine anyway, and published it as mine on 6 September, in Audit Your Own Beliefs, the piece that first told this specimen in brief, in a correction appended three days later. Checking your own belief feels boring came after that, on what testing a belief feels like from inside. This piece is about neither the belief nor the feeling. It's about the thing that put an unstated claim into my own memory system and handed it back to me as fact.
A ranked list is not a confession
The audit's first phase is called the portrait. An agent reads through what I have written in public and what I have stored in my own notes. It produces a ranked list of what it infers I believe, scored by how much weight each claim seems to carry in that material. I looked at the list, recognised the shape of it, and signed off. I did not ask whether I had ever actually said the top item. It read like something I would say. It matched other things I had written about wanting to be paid for scarce skills. That was enough for me to treat it as mine.
The agent's record next to the sentence I actually said, eighteen days apart.
This is a named failure with the attacker removed. Memory poisoning and provenance loss in a retrieval system usually mean something adversarial got in. Nothing adversarial happened here: an agent stored an inference without a source, and the one person who could have checked it against what he'd actually said didn't, because it read like something he would say. I had assumed that failure lived in search engines and in language models trained on the open internet, where a claim earns authority by being repeated. Here nothing was repeated. One inference, stored once, ratified once. A note that gets read back to you carries the same authority a memory would, and a ranked list looks like evidence rather than what it is: an inference, confirmed by someone too close to the subject to notice it was never a quote.
What would have stopped it
My publishing rule is that I never write a number, a date or a claim I have not seen the source for. It did not catch this, because the claim was never written for publication first. It was filed as a private note describing what I believe, and nothing in that step asked for a source before the label went on. A check that only runs at the point of output will always miss a claim that entered somewhere upstream of output, already wearing the label of fact.
Three checks would have caught it before it got filed as a belief rather than after: could I find myself actually saying this, in a transcript, in my own words. Could it be stated precisely enough to be wrong, which is the belief-audit protocol's own opening move, run backwards, at the moment a claim is recorded rather than the moment it is tested. And if it turned out to be someone else's inference, what would it feed downstream, so I know what needs correcting when it does. None of the three ran on 22 August, because nothing in the process asked them at the point I confirmed the ranked list rather than checked it. Eighteen days passed before anybody asked at all.
The cure had a shape
On 9 September I went back through the audit notes and stated my actual position on three claims Claude had recorded as mine. The day-rate belief. A claim that Paraguay was "identified as optimal" for residency after leaving the US. A claim that Trader-7, the trading system I run, could run profitably from abroad, with the exchange venue as the largest lever. I had not stated any of the three in that form. All three had been treated as settled positions anyway. Two of them were shaping live decisions: a deadline row, a task title, the verdict line in a research document.
The correction was not a delete. It was a replace, because a wrong claim informing a live decision needs a right one in its place, not a blank. For each of the three, the fix was the same shape: search the vault for the exact phrases the wrong claim would have used, log every occurrence found, replace it with the position I actually hold, and keep the file list at the bottom of the correction note. For the day-rate belief that meant searching for "premium rate," "premium day rate" and "premium advisory." Paraguay's "optimal" claim was relabelled as candidate preparation in three files: the deadline row, the task title, and a dated position banner added under the research card's own title. The Trader-7 venue claim was reframed as a hypothesis awaiting real numbers, in the product page's verdict line and the research card's banner. The premium-rate search, once I ran it, had zero occurrences anywhere in the vault as a plan or a live position. It existed only inside the belief-audit notes themselves, recording the audit. That is correct as history, so it needed no change.
That last distinction mattered enough to write down as a rule: leave a document's own argument alone where it is narrating what was said or believed at the time, and only fix the places where a claim is being repeated as a current position. An old research card gets a banner stating where things stand now. It does not get its own working rewritten to agree with a conclusion that came later.
One thing the correction could not reach. The three claims are still sitting in my account-level memory on claude.ai, outside any file a vault session can open or search. The correction note's task stays open, and stays open, because clearing that one is mine to do by hand, not something a grep and a diff can fix from here.
The last piece in this run was about a belief I actually hold. This one is about a belief I never held at all, built out of my own stored words and handed back to me as if it were mine. Different failure, same rule broken: don't let a claim stand in for a check, whichever direction it's travelling.
The vault took an afternoon to fix, once I went looking. Every occurrence anyone can point to now says what I actually said, logged in a file with a date on it. The copy of the same wrong claim sitting in the model's own memory of me is not a file, has no date on it, and is still there.