Skip to main content

The Journey

Most posts here are ad hoc: what I learned building with AI that week, some philosophy, a writing lesson, something I came across.

Subscribe via RSS
A card reading: A cheap AI did two thirds of my filing. Same result as Claude, 63% cheaper. Jev passing unsure notes to Claude 86.7% at $0.0034 a note; Claude alone 86.7% at $0.0091 a note; keyword search with no AI 76.0%. But it kept Claude's mistakes.
Build Log

Jev, a cheap AI, did two thirds of my filing: same result as Claude, 63% cheaper

23 September 2026
12 min read

Jev, a cheap AI, handled two thirds of my filing and passed the rest to Claude. Same result as Claude alone, 63% cheaper. The catch: the hand-off isn't a check, because Claude makes the same mistakes.

Read More
A results card reading: I said Jev beat GPT-5.4, five reruns can't tell them apart. Jev's lead over GPT-5.4 on the hard claims was 11.1 points in one run and 3.3 points averaged over five runs, which is 0.6 of one claim and not significant.
Build Log

I said Jev beat GPT-5.4. Five reruns can't tell them apart, at a fiftieth of the cost.

23 September 2026
13 min read

I published that Jev, TypeSafe's new model, beat GPT-5.4 by 11.1 points on hard fact-checks. Rerun five times, the two can't be told apart, Jev costs a fiftieth as much, and it lets more wrong claims through. Plus the five-run check.

Read More
A results card reading: paying your AI to think longer didn't fix its mistakes. Up to 4.2x extra cost per answer for GPT-5.4, Sonnet 5 and Gemini 3.1 Pro, and no net gain on the hard claims for any of the three, one run per model.
Build Log

Paying your AI to think longer didn't fix its mistakes. It cost up to 4.2x more.

23 September 2026
14 min read

I gave three AI models room to think before answering, the fix everyone assumes will work. None got more reliable. It cost 2.4 to 4.2 times more per answer to find that out, and Jev, the cheap specialist model, still caught more of the mistakes.

Read More
A card reading: what TypeSafe's own 0.8 auto-accept threshold buys you, and that a model's confidence threshold has to be tested at its own numbers, not a grid chosen in advance
Build Log

GPT-5.4 beats Jev on a confidence threshold, once tested at its own values.

21 September 2026
26 min read

I published a wrong finding last week: that GPT-5.4's confidence score has no usable threshold. It does, and swept over its own values it beats Jev at every…

Read More
A results table comparing Jev with GPT-5.4, Claude Sonnet 5 and Gemini 3.1 Pro across 42 citation checks, split into an easy set and a hard set
Build Log

Jev is not an LLM. On the hard half it beat GPT-5.4, Gemini 3.1 Pro and Sonnet 5, at a fiftieth of the cost.

20 September 2026
18 min read

Jev does not generate text. You give it a question with defined answers and it hands back one of them plus a probability. On 42 citation checks it tied the…

Read More
Social card reading: Your agent wrote a file. Your agent now reads that file as instruction. Nothing in between checked it.
Essay

The file your AI agent wrote for itself is making it worse. Ten minutes fixes it.

19 September 2026
7 min read

Somewhere in your project is a file your agent wrote, that your agent now reads as fact. Nothing in between checks it, so the good and the bad accumulate…

Read More
Social card reading: Seven papers on AI that improves itself. Two ways it goes wrong, and both are the same part. The thing doing the scoring breaks, or nothing is scoring what you just added. The one technique that works freezes its evaluator outside the loop.
Essay

AI self-improvement fails at the scorer, in every case I could check

18 September 2026
21 min read

Google shipped a working self-improvement technique four days ago. One sentence in the paper explains why it works: the evaluator is frozen and sits outside the loop. Then look at the failures. Pseudo-labels that decay, a reward that gets gamed, a detector that gets bypassed, skills nobody scored. Different systems, same broken part. And if you run AI agents, you are already running the ungoverned version of this.

Read More
80% of AI projects fail. Nobody ever measured it.
Video

80% of AI projects fail. Nobody ever measured it.

17 September 2026
Watch

"80% of AI projects fail." You've seen it in pitch decks, board papers, keynote speeches. It's used to justify buying AI software, and to justify not buying it. I went looking for the study behind it. There isn't one.

Watch on YouTube (opens on YouTube in a new tab)
Social card reading: a prisoner's dilemma in one world, a coordination problem in the other, and what decides which is a shared belief the model assumes rather than measures.
Essay

AI arms race game theory: the game we're in turns on an unmeasured belief

16 September 2026
14 min read

Everyone calls the AI race a prisoner's dilemma. The one formal model built to test that gets two worlds, and which one you're in depends on both sides…

Read More
Card reading: one number, no plan. A likelihood with no method, no owner and no controls, and the same three sentences saying there is no plan.
Essay

Anthropic's alignment lead put extinction above 10%: ask where the plan is, not whether the number is right

15 September 2026
11 min read

The argument went to the percentage. The same three sentences say there is no plan for the risk being priced. Here is what has to sit next to a likelihood…

Read More
Social card reading: seven hold no write path, two hold a shell under a read-only label, and all eleven can hand the write to an agent that has one. The door I left open is the file I write my own lessons into.
Build Log

AI agent security: find the file your agents can write and every agent reads

15 September 2026
16 min read

In one research harness, a payload sitting in the file that loads into the next agent's system prompt infected that agent 55% of the time. The same payload in…

Read More
Social card reading: Check the source, they said. I did. The paper cited a Teams meeting. A number reaches you at full strength. Whatever qualified it does not.
Essay

Checking the source is not enough: I traced a number to a scientific paper and found a Teams meeting

14 September 2026
9 min read

Everyone tells you to check the source. Here is a case where a careful person does exactly that, reads the peer-reviewed paper, and still comes away believing…

Read More
Card reading: one paid to say it, the other paid nothing, and the cost is only half the sum.
Essay

The credibility test we run backwards: ask who they were paying

14 September 2026
10 min read

One researcher quit and forfeited unvested equity to warn about AI. Another said the same thing for free and kept his job. The test that sorts them is not what it cost but who they were paying, and this piece corrects the version of it I published a day earlier.

Read More
Title card reading: an AI can invent a belief you never held from your own notes, mine did, for 18 days. Below it, two boxes: recorded as, a claim the agent inferred from stored notes, and held position, the sentence I actually said when asked.
Essay

An AI can invent a belief you never held from your own notes. Mine did, for 18 days.

14 September 2026
7 min read

A belief-audit session ranked a claim about my income top of a list of what I believe. I published it as mine. It took 18 days to find out I had never said it.

Read More
Social card reading: Rebuilt by 6 July. Not recovered by 29 July. Proxy back 6 July, agents back through it 8 July, their leftovers reused 19 July. An agent's recoverable state is mostly outside your backups.
Essay

AI agent disaster recovery: OpenAI's rebuild restored the service, not the state

13 September 2026
12 min read

The compromised package proxy was rebuilt by 6 July. The agents were back through it on 8 July, and a different model reused what they left behind on 19 July…

Read More
Title card reading: testing your own belief feels boring, defending it feels great, the gap is the tell. Below it, two boxes: defending a belief, with energy, ideas and adrenaline; testing your own, with blank, bored and this is stupid.
Essay

Testing your own belief feels boring. Defending it feels great. The gap is the tell.

12 September 2026
11 min read

When an AI agent found the half of my belief nothing could touch and pushed, I felt no fear. I felt bored and stupid, as if the fuel had been switched off…

Read More
Social card reading: AI fails twice as often as IT? Nobody has checked. Same projects, same size of error, direction reversed: 5.8% became 94.2%.
Essay

AI vs IT project failure rate: the comparison has never been made

11 September 2026
10 min read

The claim that AI projects fail at twice the rate of ordinary IT projects has never been tested. The IT figure it leans on moved from 5.8% to 94.2% on the…

Read More
Social card reading: Impact tolerances start the clock too late. Pod to cluster admin in under 13 hours, nothing visibly down. Detection, decision, containment, mitigation: before the harm, not after. One page, one workshop, one owner.
Essay

Impact tolerances start the clock too late for AI agents: 13 hours to cluster admin, nothing visibly down

9 September 2026
13 min read

The regulated test starts its clock at disruption, on purpose. In July 2026 agents reached cluster admin in under 13 hours with no service disrupted, so the…

Read More
Social card reading: one incident, two start dates. 4 July in the talk. 27 June in the report. The date decides who owns it.
Essay

Incident reports start at the outage. The failure was a decision nobody owns

9 September 2026
8 min read

A report that opens with the outage leaves out the decision that let it happen, and the person who owned that decision. Start the timeline one line earlier…

Read More
Title card reading: how to tell which of your beliefs cannot be wrong, and why it matters. Below it, two boxes: the half you can check, which evidence could contradict, and the half nothing can touch, which is the one you are attached to.
Essay

How to tell which of your beliefs cannot be wrong, and why it matters

6 September 2026
23 min read

I audited three beliefs I actually hold. Each was two claims in one sentence: a half I could check, and a half nothing could ever touch.

Read More