Skip to main content
Jamie Watters

China's social credit system never had a machine behind it. Could AI build one in the West?

Jamie Watters

Operational resilience and AI delivery practitioner.

Published: 30 October 2026•6 min read
#social-credit#china#agi#truth#reputation
Social card. Kicker: China's social credit system. Headline: never had a machine behind it, could AI build one in the West? Below: in one Chinese city, good deeds were written down with a pen. The West already runs pieces of it, from credit scores to Uber ratings.

Ask most people what China's social credit system is and you'll get the picture I had. A camera catches you crossing the road on a red light, and points come off your score. Park in the wrong place, and more go. Let the score fall far enough and you're shut out of services.

It's Orwellian. And most of it doesn't exist.

There's no single score for every citizen. "Contrary to popular belief, there's no central social credit score for individuals," MIT Technology Review reported in 2022. "And frankly, the Chinese central government has never talked about wanting one."

But part of my picture is real. And the reason it's real is the part I can't stop thinking about, because I think AI will one day apply the same logic to all of us.

What China actually built

Refuse to comply with a Chinese court's judgment, and you can end up blacklisted. Your name, your identity card number and the facts of your case can be published online. You can be banned from buying plane and high-speed train tickets (China Daily, 2019).

The scoring did happen, but locally. In 2013 Rongcheng, a city of half a million people, gave every resident a base score of 1,000. Under a 2016 rule, since overhauled, "spreading harmful information on WeChat, forums, and blogs" cost 50 points. One resident lost 950 points in three weeks for repeatedly distributing letters online about a medical dispute (MIT Technology Review, 2022).

None of it is the AI machine people imagine. "There is no known instance in which automated data collection leads to the automated application of sanctions without the intervention of human regulators," Kendra Schaefer of the consultancy Trivium China wrote, as quoted by MIT Technology Review. In Rongcheng, "information gatherers" walk the village writing down their neighbours' good deeds with a pen.

So the dystopia most people pictured needed a machine China never built. Hold on to that.

Two panels: the social credit people picture, and what China built. The picture: one number for every citizen; a camera sees you cross on red and points come off; shut out of services. What exists: no central score for individuals; refuse to comply with a court and you can be blacklisted, named online and banned from planes and fast trains; local pilots such as Rongcheng, 1,000 points to start, good deeds written down with a pen.

The scary version needed a machine. China's runs on courts and clerks.

The West already runs it in pieces

Here's where I may lose you. I don't think the logic behind social credit is mad. I think it's the logic we already live by. We just don't call it social credit.

In the US, your credit score helps decide whether you're offered a mortgage, a credit card or a car loan, and it's used to set the interest rate you pay. Insurers use it, and so do landlords screening tenants (Consumer Financial Protection Bureau).

Uber's own rules say riders who don't meet the minimum average rating for their city may lose access to all or part of its platform (Uber Community Guidelines).

Three cards headed: the West runs it in pieces, we don't call it social credit. Your credit score helps decide if you get a mortgage, a card or a car loan, and sets the interest rate you pay. Insurers use your credit score too, and so do landlords screening tenants. Uber riders below their city's minimum average rating may lose access.

Three records the West already reads, and acts on.

The logic I can't argue with

So here's my question. If you had resources to hand out, would you give more to someone who'll use them well, or to someone who'll abuse them? If you run a company, you have a fiduciary duty to put resources where they'll give shareholders the best return. We don't call that tyranny. We call it the job.

I think a world with AGI works the same way. Resources will go to the people who'll use them well: the ones who add to everyone's wellbeing and leave society better than they found it. And I think an AI government would apply exactly the principle a good CEO applies. The difference is that it would apply it perfectly.

That's the machine China never built. One that joins every record up and acts on it, with no clerk and no pen.

And it cuts both ways. A machine that can see who abused what they were given can also see who used it well. I think the second group wins: their reputation and their value go up. Even today's crude version works like that. Usually, the US consumer regulator says, a higher credit score makes it easier to qualify for a loan and "may result in a better interest rate or loan terms" (Consumer Financial Protection Bureau).

A chain of three allocators under the line: one rule, give more to those who'll use it well. A lender uses your credit score to set your interest rate. A CEO puts resources where they give shareholders the best return. An AI government, marked as the author's belief, not a fact: the same principle, applied perfectly, to every record.

The same rule at three scales. The third is my belief.

The strongest case against me

The people who study China's system would stop me there, and they'd be right to. Who decides what "use them well" means?

Shazeda Ahmed is a Princeton researcher who spent several years in China studying how tech companies are involved in it. She put it in one line: "this was always about making people honest to the government, and not necessarily to each other" (MIT Technology Review, 2022).

A bad rule run by clerks with pens misses people. A bad rule run perfectly misses no one.

Two panels under the question: who decides what use it well means? A bad rule run by clerks with pens catches some people in a row and misses others. The same rule run perfectly catches every one. An illustration, not data.

Perfect execution only helps if the rule is right.

My answer

If the AI is ethical, I think it would be the best placed to decide. Look at who decides now.

The trouble with most social programmes is that people decide who gets what, and people have biases and blind spots. Sometimes they decide without critical thinking, objectivity or high ethical principles. Remember Rongcheng? It was the city that decided "spreading harmful information" online should cost 50 points (MIT Technology Review, 2022). People wrote that rule, blind spots and all. Nothing does more harm than do-gooders, and sometimes people are more concerned with appearing good than with doing good.

The obvious comeback is: who decides whether the AI is ethical? I don't think that's the key question. What matters is how the AI is trained, and I think we can train models for the highest possible ethical behaviour. Unfortunately, we've had much less success training civil servants and politicians.

So build a record you'd be happy to have read. I've already placed my bet on it being read. In The Truth Wager I argued that a superintelligence will one day know everything we ever said, and that with what's coming you'd be a fool to bet against the truth. This is the other half of that bet. When the record gets read, I don't think it will only be judged. I think it will decide what you're given.

Sources

No paywall, no sponsors. If this saved you some time, you can buy me a coffee.

☕ Buy me a coffee

Get the next one in your inbox

I build with AI in the open and write up what held and what didn't. Real numbers, the failures before the wins.

Share this post

← Previous Post

The Truth Wager: would you bet against the God of the singularity?

Pascal's wager for the AI age. I believe a superintelligence will one day read everything we said. Hype is winning now. Why I still bet on the truth.

← Back to all posts