How to kill a trading strategy in 90 minutes: index deletions and spinoff stubs, tested

I was about to spend $630 on market data. I didn't, and the reason is a single page of text I wrote before I looked at anything.

The idea was reasonable. I already run an agent that trades crypto on paper, so the obvious next move was equities. Not price prediction, which is the most competed forecasting problem on earth and one I have no business entering. Something narrower.

Big institutions are barred from certain trades by their own rules. A fund running half a billion cannot meaningfully buy an eighty-million-pound company: even if it bought the entire thing, the position would be too small to matter. Index funds have it worse. A fund tracking the S&P 500 must hold exactly what is in the index, so when a company drops out, the fund has to sell it, on a known date, at whatever price it can get. It has no discretion at all.

That leaves prices that are wrong and stay wrong. Not because they are hard to spot, but because almost nobody is permitted to take the other side of the trade.

An AI agent's real strength is reading unstructured documents at volume, cheaply. Roughly seven thousand filings hit the SEC's EDGAR system on a normal day and almost none of the small ones get read, because reading them doesn't pay anyone's salary. That is a coverage problem, not a prediction problem, and coverage is exactly what these things are good at.

Good premise. It was dead in under two hours.

The page I wrote first

Before touching any data I wrote a pre-registration: a document naming the exact test, the exact entry and exit, what I would compare the result against, what the trading would cost, and the specific result that would end the project.

That last field is the one that matters. Mine had four conditions, all of which had to hold:

The average trade must beat the market by at least 1.0 per cent after costs. Beating the market is the benchmark: if a stock rises 3 per cent and the market rises 3 per cent, I have discovered nothing.

The result must be strong enough not to be luck. Statisticians measure this with a t-statistic, which is just signal divided by noise: how big the average result is, relative to how much the individual results scatter around it. I set a floor of 2.0, which is the conventional line for "unlikely to be an accident."

It must hold on data I had never looked at. Split the history in two, design the rules on the first half, then test on the second half with the rules frozen. Same principle as a holdout set in machine learning, and for the same reason: anything will look brilliant on the data you tuned it against.

And there must be at least twenty tradeable events a year, or the sample is too small to tell edge from luck no matter how good the returns look.

I also worked out what the trading itself would cost. Every round trip, buying then selling, loses about 1.55 per cent of the trade value: the gap between the price you can buy at and the price you can sell at, the amount the price moves against you as your own order fills, and commission. So the 1.0 per cent bar sits deliberately above the cost of playing, not level with it.

Then I wrote down the arithmetic of my own optimism. I had six candidate ideas. Statistical tests usually accept a one-in-twenty chance of a false alarm. Run six of them and the odds that at least one throws a false alarm are roughly one in four. If I tested all six and celebrated the one that passed, the most likely explanation for my success would be that I had tested six things.

The header line on the document says: changing anything below after seeing results invalidates the test.

Test one, and the moment it nearly worked

The first idea was index deletion. When a company drops out of the S&P 500, every fund tracking that index has to sell, on a known date, regardless of price. Forced seller, no obliged buyer. Textbook.

I pulled every deletion from the S&P 500, the MidCap 400 and the SmallCap 600 since 2014, using free public sources, and classified all 1,219 of them by why the company left. Acquired, bankrupt, moved to one of the other S&P indices, or dropped for having got too small or too thinly traded.

Most deletions turn out to be useless. Of the 1,219, some 363 were acquisitions, and once a takeover is agreed the share price sits pinned to the offer price, so there is no dislocation to buy. Another 616 were moves between indices, where a company falls out of the 500 and into the 400. My pre-registration had already excluded those, for a good reason: the funds tracking the 400 must buy at the same instant the funds tracking the 500 sell. The forced seller has a forced buyer sitting opposite. No dislocation there either.

That left 160 genuine drops. Call it 14.9 a year over the test period, against a floor of 20.

Here is the interesting part. When I set up the counting job, I wrote the brief carelessly and defined a tradeable event as an index move or a drop, contradicting the pre-registration I had written an hour before. Under my sloppy new definition the count came out at 77.4 a year. Comfortable pass, every single year.

Narrow definition: 14.9, fail. Wide definition: 77.4, pass. Same data.

If I had written the wide brief first, or if I had been free to choose after seeing both numbers, I would have taken the 77.4, spent the $630, and lost several weeks on a strategy whose central mechanism does not exist. I would have had a perfectly good story about why index moves count. I would have believed it.

The gap between the two documents was about an hour, which tells you something useful. Pre-registration does not work because time passes and cools you down. An hour is not a cooling-off period. It works because of the order the information arrived in. The rule was fixed while the answer was still unknown, so there was no way to tune it toward a result I wanted. Sixty minutes is enough for that, and sixty days would not help if you wrote the rule after seeing the data.

Test two

Spinoff stubs. When a company spins off a division as a separate business, existing shareholders are handed shares in the new, smaller company. Those shares are the stub. Index funds holding the parent end up owning something that usually fails their own inclusion rules, so they have to sell it mechanically. No analyst covers it. It is too small for most funds to buy. Same shape as the first idea and cleaner.

Before a company can spin a division off, it has to register the new business with the SEC on a document called a Form 10. I parsed all 4,211 Form 10 filings since 2014, then confirmed which spinoffs actually happened by checking whether the new company went on to file an annual or quarterly report, because registering is not the same as going through with it. Then I filtered to US parents listed on a real exchange.

13.1 completed spinoffs a year across 2018 to 2025. After hand-reviewing every one of the 105 events and removing bank holding company reorganisations, a post-bankruptcy relisting and a couple of foreign parents, it fell to 11.6. Against a floor of 20.

Worse, the trend is one-directional. 2014, 2015 and 2016 all cleared 20. Not one of the nine years since has. The series is not bouncing around the threshold. It sits below it and it is falling.

Test three, and why I didn't run it

The next idea was 13D filings. When an investor buys more than 5 per cent of a company with the intention of pushing for change, they must disclose it publicly within days, on a form called a 13D. Event volume was never going to be a problem. Several hundred a year.

That is precisely why I stopped.

Volume and competition are not independent variables. They are two readings of the same thing. Any situation that occurs often enough to support a monitoring business has one built on it already, and 13D watching supports several: free alert services, dedicated newsletters, entire funds that do nothing else. The filing is public and reflected in the share price within minutes.

So passing the event-count gate would not have been evidence the idea was good. It would have been evidence the idea was crowded.

Which reframes both failures into one finding. The constraint that creates the mispricing is the same constraint that limits how often it happens. Institutions are barred from these situations because the situations are small and rare. Take away the small and rare and you take away the reason nobody else is there. The uncontested set is real. It is just too thin to build a system on.

That is an architecture-level result, not two unlucky ideas, and it kills the project rather than sending it back for a redesign.

What I'd take from this if I were building something else

Write the number that ends the project, before you look. Not a vibe about what would disappoint you. A number, on a dated document, with a line saying that changing it later invalidates the test. You will not feel the value of this until the moment it costs you something, which is the moment it pays for itself.

Put your cheapest disqualifying test first. I never measured a single return. Both ideas died on a head count of how often the situation even occurs, which is countable from free public filings in an afternoon. Most people build the returns engine first because it is the fun part, then discover the sample was never large enough to support it. Ask what could kill this for nothing, and do that.

When a definition change would flip your result, stop. That is not a modelling decision. It is the exact moment you are about to fool yourself, and it never announces itself as such. It arrives dressed as a reasonable methodological refinement.

Free data lies by omission. Most free price feeds only carry companies that still exist. The ones that went bust or got taken over have quietly vanished from the record. Test any small-company strategy on that data and the results come back positive every time, because the failures were deleted from the sample before you looked. If you are testing something that selects for companies in trouble, this is not a minor caveat. It is the whole result.

Count how many things you are testing. Six ideas, one-in-four odds that at least one looks like a winner by pure chance. If you do not track the count, your fourth attempt will produce something beautiful and you will have no way of knowing it is noise.

The bit about AI that actually matters

The agent did real work here. It parsed 4,211 filings, found and fixed three bugs in its own classification logic that were silently dropping genuine events, and refused a brief I had written badly because that brief contradicted a document written an hour earlier.

Note what did the refusing. Not the model's judgement. The document.

If you are handing serious decisions to an agent, the control is not a better prompt or a smarter model. It is a written commitment made before the work starts, sitting outside the loop, that the agent can be pointed at and that you cannot quietly edit once you have seen which way the answer fell. Agents are very good at building a case for whatever you have just implied you want. Give them something fixed to check against instead.

The other honest note: the capability was real even though the strategy wasn't. A pipeline that reads thousands of regulatory filings, classifies them, and tells you plainly that the answer is no, is worth having. It just turns out to be worth more pointed at research than at trading.

Under two hours. Zero pounds. One dead idea, and a reusable page of text that killed it.