What is an overfit trading filter?

One whose apparent ability to separate good trades from bad comes from the specific period it was fitted on rather than from anything durable. Ours showed aligned picks winning ~44% against ~28% on an in-sample time split, and that spread held across sub-periods — the usual check for exactly this. Live, it inverted: ~13% for the cohort it kept against ~40% for the cohort it discarded.

Why is an inverted filter worse than a signal that stops working?

A signal that stops working adds noise — you take trades you should not have and the average degrades roughly in proportion. A filter that inverts removes the better cohort and keeps the worse one, so the book ends up worse than if you had not filtered at all. It selects against you, and it does so more forcefully the more often it fires.

How do you catch this faster next time?

Two things. Measure the cohorts the filter separates rather than only aggregate performance — the harm appears in the comparison the filter is making, not in the headline win rate, which is too noisy to distinguish a step change from a bad run until the losses have already accrued. And run filters under a governor that re-validates them forward and disables them when they stop beating their baseline. Tying the change to a deploy rather than a date is what finally made ours undeniable: the switch-on coincided with a hit rate falling from 24.5% to 13.6%.

Why keep computing a signal you have switched off?

So it stays measurable. The flag is still stamped on every pick while nothing acts on it, which lets a rolling walk-forward keep testing the hypothesis honestly and lets it be re-enabled if it ever earns its keep out of sample. Deleting the measurement along with the behaviour would mean never finding out whether the idea was real.

An Overfit Filter: In-Sample 44%, Forward 13%

Most writing about overfitting is about a strategy that looked good and then made no money. This is worse and more instructive: a filter that looked good, shipped, and then spent weeks preferentially selecting the trades that lost. It had passed the check that is supposed to prevent exactly this. Here is what it looked like, how long it took to catch, and the change we made so the next one gets caught faster.

Expectancy calculatorCheck what a filter has to do to your win rate to be worth its cost.

The signal, and why it was believable

Exchanges publish a long/short account ratio for futures: how many accounts are positioned each way. The hypothesis was ordinary — that being on the same side as the crowd at entry separates good entries from bad ones. On an in-sample time split it looked clean: picks aligned with the crowd won about 44% of the time against about 28% for those positioned against it. A sixteen-point spread is large, and crucially the split HELD across sub-periods, which is normally the check that catches a fluke.

Forward, it did not just fade — it inverted

On the live journal, aligned picks fell to roughly 13%, and about 8% over the most recent sixty. Meanwhile the small set of 'against' picks the filter was discarding won about 40%. That is not decay toward zero, which is what an overfit signal usually does. The relationship reversed sign, and the filter was gating on it in the wrong direction the whole time.

                      aligned      against
in-sample split        ~44%         ~28%      <- shipped on this
live journal           ~13%         ~40%      <- inverted
last 60 picks           ~8%           --

The gate kept the aligned cohort and discarded the against cohort.

Why a filter is more dangerous than a signal

This is the part worth generalising. A signal that stops working adds noise: you take some trades you should not have, the average gets worse, and the damage is roughly proportional to how wrong it is. A FILTER that inverts does something else — it removes the better cohort and keeps the worse one, so it makes the book systematically worse than not filtering at all. You are not merely failing to add value. You are selecting against yourself, and the more confidently the filter fires, the more harm it does.

How long it took to notice, and why

Too long, and the reason is instructive. Hit rate is noisy, so a step change reads as a bad run until enough trades accumulate to distinguish them — and by then you have accumulated the losses that made the distinction possible. What eventually made it undeniable was tying the change to a deploy rather than to a date: the switch-on coincided with a hit-rate step change from 24.5% to 13.6%. Aggregate performance metrics did not surface it. Comparing the two cohorts the filter was separating did, because that is the comparison the filter is making on your behalf.

What we changed, beyond turning it off

Turning it off is the easy part. Two structural changes matter more. First, the flag is still STAMPED on every pick even though nothing acts on it — so a rolling walk-forward can keep measuring the hypothesis honestly, and it can be re-enabled if it ever earns its keep out of sample. Killing the measurement along with the behaviour would have meant never learning whether it was real. Second, filters like this now run under a governor that keeps re-validating them forward and switches them off when they stop beating their baseline. The gate did not have that; the thing that would have caught it in days is the thing it lacked.

# Default OFF. Still stamped, so it stays measurable.
LS_ALIGNMENT_GATE = os.getenv("LS_ALIGNMENT_GATE", "0") == "1"

c2["ls_aligned"] = None if _lsr is None else (not _ls_against(_lsr, direction))
if _gate_active and c2.get("ls_aligned") is False:
    ...  # abstain — only when the flag is explicitly on

What a filter that did earn its keep looks like

For contrast, the veto we kept. Measured on 485 resolved live picks, the one stably harmful cohort was chasing: entries taken after a coin had already moved 2-5% in the trade's direction won 20.0% with a mean return of -1.78% at t = -3.68, while flat entries in a -2% to +2% dead band won 45.2% at +0.70% and t = +2.61. It was negative in BOTH time halves, which is the property the long/short gate only appeared to have. Note what it does not do: it does not require a coin to be moving our way before we will trade it. The flat cohort is the best one, and 'prefer what is already moving' is the momentum premise this journal measures as the worst ranker it has.

entry move (in trade direction)     n     win%    mean return       t
flat  -2%..+2%  (dead band)       364    45.2%       +0.70%   +2.61
already moved +2..5%              121     20.0%      -1.78%   -3.68

Negative in both time halves (t -2.64 / -2.80).

The sample caveat, stated rather than buried

The chase measurement spans three days of journal, and every record in it is a RISK_ON regime — so there is no evidence in it about how the veto behaves when risk is off. That is exactly why it runs under the governor rather than as a permanent rule. Publishing the caveat alongside the result is the difference between a finding and a claim, and the long/short gate is the reason we now insist on the distinction.

Summary

A long/short alignment filter separated winners from losers convincingly in sample — 44% against 28%, holding across sub-periods — and inverted the moment it met new data, dropping aligned picks to ~13% while the cohort it was discarding won ~40%. Because it was a filter rather than a signal, it did not merely stop adding value; it systematically kept the worse half, and the switch-on coincided with a hit-rate step from 24.5% to 13.6%. It is now off by default, still stamped so it stays measurable, and anything of its kind runs under a governor that re-validates forward. The lesson is not 'validate out of sample', which everybody says. It is that a filter deserves a stricter bar than a signal, because its failure mode is active harm rather than noise.

We Shipped a Filter That Served the Losing Side for Weeks