A backtest that passes both buying an ETF after a 20-day decline and buying after a 20-day advance has measured market exposure. Zhang documents exactly that result in his own research system. For anyone searching a grid of timing rules, this admission is the paper's most useful finding.
Who gets to decide?
In Autonomous Quant Researcher, Zhang's open-source research loop, a language model proposes trading hypotheses and deterministic code judges them. A locally served 7B coding model is the reference proposer. The current schema permits exactly two fields, hypothesis and rationale. Inside hypothesis sit a schema version, an entry minute on a five-minute grid, a bounded Boolean signal over causal observations, and one defined-risk spread, either a call debit or a put credit spread. Extra fields and out-of-range parameters trigger rejection before pricing. Under schema 3, the model writes no code and needs no sandbox. Some earlier campaigns in the table used an executable-candidate loop that required one. The founding rule is explicit: "the model is allowed to propose, and only trusted code is allowed to decide."
Each campaign fixes chronological discovery, validation and holdout windows. Before receiving any proposal, the code hashes the costs, gates and trial budget into a contract. It rejects duplicates and one-knob variants. A 50-trial budget therefore purchases 50 distinct ideas, with at most two from any descriptor family. Discovery produces a shortlist of at most three; validation divides the 5% tail by that shortlist's size. An irreversible receipt must be written before the holdout opens, once. A hash-chained ledger records every event and fails closed if edited.
The campaign table contains about two dozen entries. It covers 0DTE index options priced from traded bars, overnight SPX verticals (some using modeled prices), daily ETF rules, technical-analysis premises, Kalshi bitcoin binaries, earnings options and off-exchange volume. Almost everything fails. Two results pass pre-registered bars, and Zhang leaves both to separate papers.
The first concerns the FINRA off-exchange share of a stock's volume. Its information coefficients are negative at 1, 5 and 20 days (t between -3.1 and -5.8), and negative in every year from 2021 to 2026. The second is reversion among the day's losers on market-wide crash days. It cleared every gate in the campaign that rejected the pre-registered gap-continuation hypothesis. Zhang calls these research findings and declines to call either a strategy.
The ETF grid passed long exposure
The daily grid offers the clearest view of where the gates succeed and where they miss. Across eight ETFs, it tested 232 single-rule cells. Four passed discovery. Each used the same 20-day feature in both directions, buying after a 20-day decline and buying after a 20-day advance.
Those conditions together cover both states of the feature. When the mirror rules both earn money, holding the asset explains the return. Zhang's formulation is blunt: "A condition whose opposite passes equally carries no information."
Every acceptance test is absolute, which is how this slipped through. Edge after base costs and mean net P&L must each exceed zero. The weekly-cluster bootstrap uses 2,000 resamples clustered by ISO week; its lower bound must be positive. So must P&L at elevated costs. The code calculates a control-relative metric, then leaves it diagnostic so that "a weak control cannot rescue a candidate." By the same logic, a strong control cannot sink a candidate.
In a profitable holding window for these ETFs, a long-biased rule can beat zero without timing skill. The paper acknowledges that the gates "caught sample and stability problems but could not by themselves detect that the whole discovery window pointed one way." Its defence follows immediately: all three shortlisted cells subsequently failed validation.
Zhang changed the rule. The grid now reads a condition alongside its opposite and classifies a pair that passes as a long-side artifact. An overnight SPX 1DTE call debit spread prompted a related repair on the options side. It cleared every validation gate with a weekly Sharpe of 2.27, although the index itself supplied 56% of the validation payoff. That options family now has a pre-registered drift control. Each trade is regressed on the underlying's move over the same holding period, and the residual faces the gate.
The SPY 0DTE put credit spread exposes the same problem. After a 0.5%-of-width round trip, it earned +55.7 and +71.6 bps of width per session in discovery and validation. Market direction accounted for 13 to 30% of those returns. The bootstrap bound for its drift-neutral residual was marginally negative, and it received no promotion.
Validation caught this one
All three shortlisted ETF cells failed every performance gate in validation. Nothing from the grid was promoted. Discovery had accepted market exposure; the next window caught it. The technical-analysis campaign tells a similar story. A regression-channel premise cleared all nine discovery gates before losing 22 bps per session in validation.
The QQQ options result limits how much comfort to take from that, though it says nothing about ETF timing. On its thirteenth run, the proposer offered a rule selling the near-money put credit spread on Mondays and Tuesdays whenever credit exceeded 10% of width. Discovery net Sharpe was 2.94 on 221 trades. Validation delivered 3.34 on 43, followed by 0.26 on 72 in the holdout. Net per trade slid from +5.69% of width to +4.82% to +0.52%.
The rule survived doubled costs and passed 8 of 9 holdout gates. One gate stopped it. Expectancy dropped about 90%, and one half of the holdout window lost money net. As the paper puts it, "a loop that accepts on a validation Sharpe above 3 would have promoted this rule."
Two aspects trouble me more than the paper allows. The validation result rests on 43 trades over about six months, with a floor of 15. Zhang cites the deflated Sharpe ratio and the probability of backtest overfitting, yet I did not find either calculated for this survivor. He cites both through Gençay in arguing that statistical correction cannot substitute for structural leakage control. His own multiplicity adjustment consists of counting distinct trials and dividing the 5% tail by the shortlist size of at most three.
I also did not find how many of the twelve preceding runs had reached this same validation window. Several visits would mean repeated reads, making the 3.34 less independent than it appears.
The holdout was spent "on explicit instruction" after the frozen specification had been saved to a temporary path and lost. Before the read, a reconstructed version matched discovery Sharpe to six decimals. The contract is now persisted with the ledger.
Keeping the experiment frozen
The engineering protects against changing an experiment after its results arrive. These nulls are credible as nulls of the search. Zhang nevertheless concedes that limited power partly explains the 46-day Kalshi and 60-weekend ETF-flow nulls. Holdout floors are thin: the reference SPY mission requires 30 trades and 12 weeks, while the traded-bar 0DTE campaigns require 15 trades. Zhang asks to be judged "first by how hard it is to fool," and the paper meets that standard. He also writes that the loop "does not make true things easy to find."
Our seven-ETF replay
We could not trade the options, Kalshi or off-exchange campaigns. Our adaptation therefore retains only the daily ETF component. The paper leaves the eight ETFs unnamed and does not give the grid's stage dates or costs. Its reference SPY mission using daily data runs through 2016 to 2021, 2022 to 2024 and 2025 onward. The paper says those details reside in campaign files available on request.
We chose seven ETFs ourselves: SPY, QQQ, IWM, TLT, GLD, XLE and XLF. Our rules use moving-average crossovers at (5,20), (20,60) and (50,200), trailing momentum over 20, 60 and 120 sessions, and short reversal over 1, 3 and 5 sessions. Each rule is paired with its opposite; both directions passing excludes the pair. Positions hold long or cash in 12.5% sleeves, subject to a 42-trial ceiling.
The run did not meet that design.
We supplied no stage boundaries, so no gate was applied. The result is an unstaged replay aggregating every trial sleeve from 2020-01-01 to 2024-07-01, with 5,288 trades. Our Sharpe over that period was 0.64. Beta to SPY was 1.80, volatility was 42%, and maximum drawdown reached -78.88%. We charged commissions of $0.004 a share with a $1 minimum and recorded fills at the close. The design specified next-open fills, proportional cost scenarios of 5 and 10 bp, and an elevated-cost check. We did not demonstrate that check.
The 1.80 beta most likely explains what happened. A 120.42% total return alongside beta 1.80 and a -78.88% drawdown reads as levered long exposure through a mostly rising window. It is the same artifact that appeared in Zhang's grid. Combining our 4.0 leverage cap with summed independent sleeves is one candidate explanation for the 42% volatility and the drawdown. A single capped sleeve would probably be far tamer.
Zhang gives no daily-grid return figures, only four passers and zero promotions. All his reported Sharpes come from options. Our 0.64 falls below the QQQ survivor's 2.94 and 3.34, and above its 0.26 holdout result. These figures measure different things. His describe a selected options candidate that passed gates over 2024 to 2026. Ours describes an ungated aggregate of ETF timing rules, overlapping about five months of his options discovery window. The gap therefore tells us about our setup only.
A fair next run would put the gating framework to work on ETFs, freezing stage dates before any evaluation and using measured costs. Beating same-ETF buy-and-hold in every stage would itself be a gate.
None of the 232 ETF cells earned promotion. The four that came closest were the market itself.
Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.