AQAI QuantAI research lab for systematic strategies

Our review of the paperwe backtested it

Conformal Kelly: coverage held, growth did not

Eight ETFs, a gross cap that binds 98% of days, and a book that is mostly levered equity beta

2026-08-06 · 5 min read · US-listed ETFs traded daily using daily OHLCV/adjusted price data.

Read the paper on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

Config B Conformal Kelly Daily ETF Allocation with Horizon Ensemble and Downside Miscoverage Dial

Backtest period 2020-01-01 to 2025-10-08 · hypothetical, net of modelled costs

Why these figures are not the paper's (2)

This is not a replication of the paper (2)

  • The exact published experiment used a frozen external Kaggle price snapshot beginning in 2006; the platform would use its own US ETF daily_prices history, generally from around 2010, so exact numerical replication is unlikely.
  • The autonomous LLM-agent search and pre-registration/lockbox protocol are not necessary for implementing the final strategy; the backtest would evaluate the disclosed fixed configuration rather than reproducing the full agentic search process.

The figures below measure what we could run, not the paper's own method, so they are not evidence for or against its claim.

Our own audit found this run does not follow the paper faithfully (5)

  • deviation left undescribed by the audit (invalidates: Exact DEV and LOCKBOX performance numbers; exact turnover and coverage comparisons from the paper)
  • deviation left undescribed by the audit (invalidates: Paper DEV Config A results; Paper DEV Config B results; Paper LOCKBOX Config A results; Paper LOCKBOX Config B results; train/dev/lockbox split comparability)
  • deviation left undescribed by the audit (invalidates: If a downstream runner enforces 0.125 as an additional hard cap, paper growth, gross exposure, turnover, and drawdown predictions no longer apply)
  • Warmup vs. backtest window: the spec sets first_prediction_after_days=750 (~3 trading years) and requires W=500 landed 21-day conformal scores before qeff is defined; scores only begin landing after predictions start and each lands 21 days later, so ~750 + 21 + 500 ≈ 1271 trading days (~5 years) elapse before a fully-formed per-asset scale exists — if the data source is loaded only from backtest_start=2020-01-01, essentially the entire 2020-01-01..2025-10-08 window is consumed by warmup and almost nothing trades; the spec never states that pre-2020 history is loaded to seed the expanding ridge and the conformal window.

1 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total 45.1%Oct 2025
Sharpe
0.41
Total Return
45.1%
Max Drawdown
-40.4%
CAGR
6.7%
Volatility
19.2%
Trades
11,446

What the paper reports for its own strategy

  • DEV Config A (2016-2021, net of 5bps/turnover, no financing): ann. net log growth 28.45%, Sharpe 1.336, max drawdown 27.68%, Calmar 1.127
  • DEV Config B (2016-2021, same costs): ann. net log growth 25.84%, Sharpe 1.386, max drawdown 20.26%, Calmar 1.376
  • DEV realized conformal coverage 0.7483 vs 0.7500 nominal
  • Lockbox Config A primary (2022-01-01 to 2024-09-20, 5bps costs, no financing): ann. net log growth +8.47%, Sharpe 0.453, max drawdown 36.6%, Calmar 0.327; secondary variant +11.47%
  • Lockbox Config B primary: ann. net log growth +7.01%, Sharpe 0.422, max drawdown 31.7%, Calmar 0.303; secondary +9.68%
  • Lockbox realized coverage 0.7450 (primary) / 0.7485 (secondary) vs 0.75 nominal

Size a position off the half-width of a conformal prediction interval and you get a risk gauge that stays honest out of sample and, on its own, does not make money once the search window ends.

The denominator is the whole trick

The construction is one substitution inside a textbook rule. For each of eight liquid US ETFs (SPY, QQQ, DIA, MDY, GLD, SLV, USO, DBC), an expanding-window ridge regression (lambda=10, refit every 21 days) maps four features (momentum over 21, 63 and 252 days, plus a 20-day EWMA volatility) to the forward 21-day return sum. The nonconformity score is the plain absolute residual. The 75% conformal quantile of the last 500 landed scores, geometrically shrunk toward an expanding anchor, becomes a scale: sigma = q_eff / 1.2816. Position size is fractional Kelly, f = 0.15 * mu / sigma^2, winsorised at plus or minus 0.75 per asset, then renormalised so gross exposure never exceeds 2.0. Wider interval, smaller bet. A calibrated width is an estimate of how wrong the forecast tends to be, and Kelly wants exactly that quantity in its denominator.

On the development window (2016-2021, 1,511 days, net of 5bps per unit turnover, no financing) the paper reports 28.45% annualised net log growth at Sharpe 1.336 with a 27.68% drawdown for the plain configuration, and 25.84% at Sharpe 1.386 with a 20.26% drawdown once a drawdown dial is added. Realized coverage was 0.7483 against 0.7500 nominal.

Why did the slow width win?

The result that earns the paper its keep is a negative one against the conformal-for-time-series literature. Every device that makes the interval adapt faster lost development growth: volatility-scaled scores -3.3pp, adaptive conformal inference -1.6pp, recency-weighted calibration -1.4pp, asymmetric CQR -1.6pp, pooled -2.5pp, Mondrian -2.4pp. A sigma frozen on pre-2016 residuals for six years (0.2669) still beat a rolling residual standard deviation (0.2476). At matched gross the conformal quantile beat the plug-in standard deviation by 2.1 percentage points a year and 0.08 of Sharpe, with mean absolute deviation sitting between them.

The explanation is mechanical and I find it convincing. A sizing map integrates the interval, so it is charged for the estimation variance of q, not rewarded for local sharpness. The growth ordering across estimators is their tail-sensitivity ordering: the standard deviation's influence is quadratic in the residual, the MAD's linear, the quantile's bounded. Understating fat-tail sigma keeps more weight on the profitable assets, which pays only because the next fact holds.

A cap doing the heavy lifting

Pre-cap gross leverage averages 4.28 and the 2.0 cap binds on 97.7% of development days. So kappa is inert, correlation is inert, and constant de-levering changes nothing. The book sits pinned at maximum gross, on the rising part of the growth curve, essentially always. A drift-only forecast (pure intercept, zero features) already reaches 82% of the final result, and the four equity ETFs supply +0.237 of Config A's +0.312 total arithmetic contribution. This is substantially a levered long-equity book. A post-hoc four-equity 2x bar beat Config A on development growth (0.2934 versus 0.2845), though at a 60.9% drawdown against 27.68%. The winsorisation clip is load-bearing too: renormalising before clipping ("water-filling") costs 4.0pp.

What we ran, and what it showed

Two things about our build before any of our figures. We could not replicate the exact experiment: the paper uses a frozen Kaggle price snapshot starting in 2006, while we ran on daily ETF close history that generally begins around 2010, so exact numbers were never on the table. And we did not reproduce the autonomous agent search or the sealed-lockbox protocol, which are not needed to trade the disclosed configuration; we evaluated the fixed Config B rule directly.

Our backtest covers 2020-01-01 to 2025-10-08 and compounds at about 6.7% CAGR with a Sharpe of 0.41 and a 40% maximum drawdown. Those are our numbers, from our one automated pass. Set against the paper's own figures, they are nowhere near the development Config B headline (25.84% growth, Sharpe 1.386, 20.26% drawdown) and land almost exactly on the paper's own sealed 2022-2024 lockbox for Config B (7.01% growth, Sharpe 0.422). Our window is dominated by the same 2022-2024 regime the authors sealed away, where they too reported growth failed to transfer, so the overlap is roughly what you would expect. Two open contradictions in our setup likely widen the gap further. The warmup arithmetic (750 days plus 21-day label landing plus 500 landed scores implies roughly 1,271 days before a fully formed scale exists) means much of our window trades on immature sigma if it was seeded only from 2020. And a stray 0.125 position cap in our framework metadata, if it bound instead of the paper's 0.75 winsorisation, would have kept us well below the cap-saturated regime the authors say drives everything. Our 40% drawdown, deeper than the paper's lockbox 31.7%, is the piece I can pin least: one candidate is that our equity curve straddles both the 2020 and 2022 selloffs while the dial is documented to sell the bottom during rebounds. We cannot fully close the gap from what we can see, and a thin result on a different window is evidence about our implementation first.

The dial as a risk gauge

Using the trailing one-sided downside miscoverage rate to raise growth failed across 40-plus configurations, because the signal peaks after a shock and sells the rebound. As a leverage dial it instead cut the development drawdown from 27.68% to 20.26% while lifting Sharpe to 1.386, beating all 40 circular-shift placebos on drawdown (p=0.024), and a constant-leverage control at matched gross left the drawdown unchanged at 27.68%, so the effect is timing rather than de-levering. Out of sample the calibration transferred (0.745 against 0.750) and the economics did not: both configurations ranked last of eleven entries on Sharpe and Calmar, and at a 4% financing charge Config A's lockbox growth falls to roughly +4.7% a year, below every unlevered passive bar. The width is a clean, slow, per-asset noise estimate. Turning it into wealth under a binding cap is the part that did not survive contact with 2022.

Worth reading for the eight refuted hypotheses and the reason slow scale beats fast scale in a sizing map. Not worth trading as printed, which the author says plainly.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

For each ETF independently:
  Compute close-to-close daily returns.
  Build features from close history only:
    momentum over 21, 63, and 252 trading days
    EWMA volatility with 20-day span

On each refit date after warmup:
  For each forecast horizon H in {12, 16, 21, 27, 34}:
    Train an expanding-window ridge model per ETF.
    Truncate training labels so every forward H-day return has landed by the prediction date.
    Forecast H-day arithmetic return.
    Rescale forecast to a 21-day reference by multiplying by 21 / H.

For each ETF/date:
  Ensemble horizon forecasts using weights proportional to 1 / q_h^2.
  Compute landed 21-day forecast errors only.
  Estimate conformal scale qeff from a 500-score rolling per-asset quantile with 0.3 expanding-anchor shrinkage.
  Convert qeff to sigma_hat = qeff / 1.2816.
  Set raw fractional Kelly weight = 0.15 * mu_hat / sigma_hat^2.
  Winsorize each ETF weight to [-0.75, +0.75].
  If gross exposure exceeds 2.0, scale all weights down proportionally.

Staggering and risk overlay:
  Average the target books formed over the last 5 trading days.
  Measure pooled downside interval break rate over 21 days.
  Apply multiplier m = clip(1 - (d - alpha/2) / (alpha/2), 0.25, 1.0) to the full book.

Execution:
  Rebalance daily at the close using observed close prices, with the specified one-trading-day lag.
  Charge turnover-based transaction costs and skip fills where required close data is missing.