Skip to main content

AI risk and resilience

AI risk and resilience: agent incidents, operational resilience, controls, governance and the law.

Social card. Headline: The AI research error that gets filed isn't bias. It's the invented fact. Line below: Four models, one question, every claim held against the documents. Footer: jamiewatters.work.
Essay

I tested four AI chatbots for political bias. Gemini invented a plan to scrap the stock market

10 November 2026
8 min read

I tested four AI models for political bias on two divisive subjects and held every claim against the documents. Bias was not the worst problem: one invented policies and stated them as fact. Here is the check.

Read More
Social card. EU AI Act for businesses: three questions that tell you what you owe and when. Are you building it or using it? Is what you use it for on the list? Does anyone outside the business see what it makes? Most firms that use AI owe little today beyond helping staff understand it; AI in hiring, credit or life and health insurance pricing owes more from 2 December 2027.
Essay

EU AI Act for businesses: three questions that tell you what you owe and when

3 November 2026
15 min read

Most businesses that use AI are deployers under the EU AI Act. Today that mostly means helping staff understand the AI they use. AI used in hiring, credit or life and health insurance pricing owes more from 2 December 2027. Three questions sort it, with the UK, US and Korean rules alongside.

Read More
Social card. Headline: Claude's new AI agent inherits every app you've connected. Mine could read my API keys. Supporting line: the link's only lock was a secret web address, and the server had written it into its own log 47,818 times. Locked down and changed on 1 November.
Build Log

Claude's new AI agent inherits every app you've connected. Mine could read my API keys

1 November 2026
6 min read

I thought I was about to give Claude's new agent the link to my notes. It already had it, and the link could open my stored keys. Six checks to run.

Read More
Social card. Kicker: China's social credit system. Headline: never had a machine behind it, could AI build one in the West? Below: in one Chinese city, good deeds were written down with a pen. The West already runs pieces of it, from credit scores to Uber ratings.
Essay

China's social credit system never had a machine behind it. Could AI build one in the West?

30 October 2026
6 min read

China never built the all-seeing social credit score. The West already runs pieces of it. I think an ethical AI could run the whole thing perfectly.

Read More
Social card reading: AI companies are buying firms on borrowed billions. If it goes wrong, the firms pay. In four completed deals, the bought firm borrows the money, guarantees it or puts up its assets.
Essay

AI companies are buying firms on borrowed billions. If it goes wrong, the firms pay

28 October 2026
14 min read

When an AI buyer takes over a business with borrowed money, the lenders' claim is on the business it bought. In four completed deals that business borrows the loan, guarantees it or pledges its assets. Amex GBT already backed a $1,486m loan before its AI buyer added $1bn of notes. Checked 28 October 2026.

Read More
Social card. Kicker: posting things you made with AI? Headline: AI content labels are law in the EU now. Two questions tell you whether your post needs one. Below: is it a deepfake? Is it AI text on public affairs that no one reviewed and took responsibility for? Then you label it. Everything else, the labelling rule puts on the tool.
Essay

AI content labels are law in the EU now. Two questions tell you whether your post needs one

22 October 2026
9 min read

Since 2 August 2026 the EU AI Act has told the person posting AI-made content when to label it. Most of the marking job sits with the tool that made it. Two questions tell you when the label is yours to add, and what China, India and California ask instead.

Read More
Social card. Research paper, October 2026. AI law is splitting apart. The duties are converging. What binds now across the EU, UK, US, China and Asia, what lands by 2028, and what every organisation needs.
Paper

The world's AI laws disagree. Four duties keep turning up anyway

14 October 2026
25 min read

Where AI regulation stands in October 2026 across the EU, the UK, the US, China and the rest of Asia: what binds now, what lands by 2028, where the rules contradict each other, and the four duties that keep recurring, with different triggers and deadlines in each regime. The key dates checked at source on 14 and 24 October.

Read More
Social card: They checked the computer's debts by hand. 29% were wrong. The volume went up anyway. Cut for speed: the officer who checked whether a mismatch was real. The pace: 20,000 a year became 20,000 a week, by the minister's count. The check: officers redid 919 debts by hand and marked 29% wrong. Robodebt, Australia, 2016, rules-based, not AI.
Essay

Automating with AI? Australia cut one human check from its welfare debt system and unlawfully billed 433,000 people

4 October 2026
9 min read

Robodebt cut a slow human check to go faster. Officers then redid 919 of its debts by hand and marked 29 per cent wrong, and the volume went up anyway. Five things to do before AI takes over a process.

Read More
Social card reading: a prisoner's dilemma in one world, a coordination problem in the other, and what decides which is a shared belief the model assumes rather than measures.
Essay

AI arms race game theory: the game we're in turns on an unmeasured belief

16 September 2026
14 min read

Everyone calls the AI race a prisoner's dilemma. The one formal model built to test that gets two worlds, and which one you're in depends on both sides…

Read More
Card reading: one number, no plan. A likelihood with no method, no owner and no controls, and the same three sentences saying there is no plan.
Essay

Anthropic's alignment lead put extinction above 10%: ask where the plan is, not whether the number is right

15 September 2026
11 min read

The argument went to the percentage. The same three sentences say there is no plan for the risk being priced. Here is what has to sit next to a likelihood…

Read More
Social card reading: seven hold no write path, two hold a shell under a read-only label, and all eleven can hand the write to an agent that has one. The door I left open is the file I write my own lessons into.
Build Log

AI agent security: find the file your agents can write and every agent reads

15 September 2026
16 min read

In one research harness, a payload sitting in the file that loads into the next agent's system prompt infected that agent 55% of the time. The same payload in…

Read More
Social card reading: Rebuilt by 6 July. Not recovered by 29 July. Proxy back 6 July, agents back through it 8 July, their leftovers reused 19 July. An agent's recoverable state is mostly outside your backups.
Essay

AI agent disaster recovery: OpenAI's rebuild restored the service, not the state

13 September 2026
12 min read

The compromised package proxy was rebuilt by 6 July. The agents were back through it on 8 July, and a different model reused what they left behind on 19 July…

Read More
Social card reading: Impact tolerances start the clock too late. Pod to cluster admin in under 13 hours, nothing visibly down. Detection, decision, containment, mitigation: before the harm, not after. One page, one workshop, one owner.
Essay

Impact tolerances start the clock too late for AI agents: 13 hours to cluster admin, nothing visibly down

9 September 2026
13 min read

The regulated test starts its clock at disruption, on purpose. In July 2026 agents reached cluster admin in under 13 hours with no service disrupted, so the…

Read More
Social card reading: one incident, two start dates. 4 July in the talk. 27 June in the report. The date decides who owns it.
Essay

Incident reports start at the outage. The failure was a decision nobody owns

9 September 2026
8 min read

A report that opens with the outage leaves out the decision that let it happen, and the person who owned that decision. Start the timeline one line earlier…

Read More
Five onboarding controls listed twice: what a new human employee gets, and what an internal AI agent gets. Only the laptop appears in both columns.
Essay

Onboarding an AI agent: five controls we use for people and skip for agents

6 September 2026
7 min read

Vercel's CEO says the first thing a firm does when it hires someone is hand them a computer. Take that seriously and the agent rollout looks unfinished.

Read More
The sabotage is real. The 65% cover-up figure is from a different study. Below: Anthropic 13 August, three agents, no attacker. UK AISI 27 April, one agent, different task. 65% of 7%, four runs in a hundred.
Essay

Anthropic's agents sabotaged each other. The 65% cover-up figure is from another study.

4 September 2026
9 min read

The sabotage is real and well documented. The concealment figure attached to it comes from a different organisation, four months earlier, in a different task…

Read More
Three detections fired. Every one was correct. Nothing stopped. Below: late May never reached the responders, 27 June correctly attributed and not stopped, July correlated and nobody paged.
Essay

AI agents breached Hugging Face despite three detections. OpenAI's fix is a 30-minute rule.

3 September 2026
8 min read

Three separate teams detected the OpenAI agent activity before Hugging Face was breached. Every detection worked. Nothing stopped.

Read More
Anthropic's safety policy has never stopped a release. Below: five failures disclosed by them, a risk grade they raised themselves, and zero releases stopped.
Essay

Anthropic's safety policy has never stopped a release. Yours probably hasn't either.

31 August 2026
11 min read

The best-documented AI safety regime in existence publishes its own failures, raises its own risk grades, and has never stopped a release. Here is the test.

Read More
A finding describes a route. A sign-off closes the route. Nobody checks the capability. Below: seven actions every one correct, sixteen hours until the same capability came back, and around a hundred audit issues closed the same way.
Essay

Incident remediation: how to tell whether you closed the route or removed the capability

31 August 2026
8 min read

Seven remediation actions, all correctly closed, and the capability was back sixteen hours later. Three questions to ask before you sign off.

Read More
The principles held. The operating model did not. Below: classification can precede action, broke 27 June. Escalation outruns consequence, broke at hour 13. Remediation removes capability, broke 8 July.
Paper

The OpenAI–Hugging Face incident broke three assumptions in how operational resilience is actually run

27 August 2026
34 min read

Detection worked. Attribution worked. A correct diagnosis on 27 June was followed by a decision not to stop, and a fortnight later Hugging Face was breached.

Read More
Page 1 of 2Next →