AQAI QuantAI research lab for systematic strategies

Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysisOur backtest

FIFO coverage: 90% overall, 76% to 78% on high-VIX days

State reservoirs raise ETF stress coverage from 0.76 to 0.85; regime-weighted FIFO wins on stress interval score, within noise

2026-11-04 · 8 min read · Volatility Forecasting and Risk-Based Position Sizing · US-listed ETFs

Reviewing: Aggregate Calibration Can Mask Stress-State Undercoverage: Bounded-Memory Forecast Intervals under Recurrent Financial Stress · Ruihang Han and Li-Hsiang Lin · Read it on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

ETF11 HAR Range Risk Sizing: FIFO versus State-Reservoir Replay

Backtest period 2020-01-01 to 2024-07-01 · hypothetical, net of modelled costs

Why these figures are not the paper's (1)

Our own audit found this run does not follow the paper faithfully (8)

  • Appendix B unrounded state-bank capacity: B̃_g(η) = B_rep * p̂_g^η / (∑_{h∈{L,M,H}} p̂_h^η).: Preserve the formula as documentation but directly fix historical capacities to L/M/H = 46/31/48; do not estimate frequencies or recompute capacities. (invalidates: A claim that this allocation was selected by inverse-frequency optimization on our validation sample; direct replication of paper allocation-selection results)
  • Appendix B optional time-weighted comparison: a^{TWC}_{i,s,t} = 2^{−d_{i,s,t}/h_{1/2}}.: Documentation only; all active primary-calibrator records have unit weight. (invalidates: TWC performance comparisons are not tested by this specification)
  • Appendix B optional regime-weighted comparison: a^{RWC}_{i,s,t} = 2^{−d_{i,s,t}/h_{1/2}} exp{−γ|c(G_s)−c(G_t)|}.: Documentation only; do not implement RWC, RWC-MM or weighted cutoffs. (invalidates: RWC-adapted and RWC-MM performance comparisons are not tested by this specification)
  • The one-sided HAR coefficients are re-estimated by ordinary least squares at every target date using at most the 756 most recent valid origin–response pairs, with an intercept; forecast generation begins once the raw range series reaches index 252.: Refit only monthly on pairs completed by the preceding month-end, retain at most 756 pairs, and freeze coefficients within the month. Initialization can postpone the first fit until a permitted month-end after raw index 252. (invalidates: Exact replication of Sections 5.1–5.3 and Tables 2–4; numerical coverage, width, interval-score, memory-age and episode benchmarks; attribution of any difference solely to the paper's unchanged daily-refitted forecasting system)

4 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total 34.6%Jul 2024
Sharpe
0.85
Total Return
34.6%
Max Drawdown
-14.4%
CAGR
6.8%
Volatility
9.0%
Beta vs SPY
0.37
Trades
12,413

What the paper reports for its own strategy

  • Rare replay (paper's proposed bounded design), high-VIX interval score at nominal 90%, B=250, evaluation 2012-2025: 0.0736 ETF11, 0.0925 Stock50. FIFO comparison: 0.0872 and 0.1058. Forecast-interval metric, not a trading return; no costs applicable.
  • Rare replay high-VIX coverage, B=250, 2012-2025: 0.8478 (ETF11), 0.8619 (Stock50), against nominal 0.90.
  • Rare replay minus FIFO high-VIX interval score: -0.0136 ETF11 (95% CI [-0.0275, -0.0018]) and -0.0132 Stock50 (95% CI [-0.0286, -0.0009]), 20-day block bootstrap, not multiplicity-adjusted.

A band covering 90% of days can miss stress days at more than twice its target rate while passing the aggregate check. Han and Lin's ETF panel shows exactly that. A calibration buffer holding only recent errors covers 0.9003 of evaluation days, yet coverage falls to 0.7576 on the 156 days with lagged VIX at 30 or above. Coverage on other days is 0.9070. The high-minus-non-high difference is -0.1494, with a 20-day block-bootstrap interval of [-0.2778, -0.0376]. The fixed 50-stock panel gives much the same result: 0.9002 overall coverage and 0.7763 during stress.

The forecast is for the next day's log high-low range, a risk input. A stop rule or inverse-volatility sizer divides by the range interval's upper bound. In ETF11, misses occur on roughly 24% of the 156 high-VIX days, against a target of 10%. Some fall above the upper bound. Sizing exposure from that bound leaves the position too large on high-VIX days.

Why would old errors help?

Age determines what a first-in, first-out residual buffer discards. When stress returns after more than 250 prediction origins, about a trading year, the buffer has already evicted every stress-era error. It then supplies a calm-market quantile. Calm days dominate the sample: 2,634 low-VIX days and 730 medium-VIX days, against 156 high. Aggregate coverage can still reach 90%.

Han and Lin keep the forecasting chain fixed. Their heterogeneous autoregressive (HAR) model uses daily, weekly and monthly lagged ranges, with OLS refitted at each target date on up to 756 recent pairs. Absolute forecast error supplies the conformity score. An adaptive conformal inference (ACI) controller adjusts the miscoverage level toward the 0.10 target, using a step size of 0.005 within [0.001, 0.30]. Each bounded design can store 250 residuals. An unbounded benchmark retains the entire same-state history. VIX, lagged one observation, determines the state: L below 20, M from 20 to 30, and H at 30 and above.

Rare replay, the authors' proposed design, divides that storage budget between recent errors and state-specific reservoirs. It retains the 125 latest residuals. Each evicted error goes into the reservoir for the VIX state in which it was issued. L, M and H receive capacities of 46, 31 and 48, allocated inversely to their frequencies during validation from 2009 to 2011. A query draws only on the recent queue and the reservoir for the current state.

The data are unadjusted Yahoo Finance OHLC for 11 ETFs and 50 large US stocks. Warm-up covers 2005 to 2008; evaluation spans January 2012 through December 2025, giving 3,520 dates per asset. The paper contains no P&L. It evaluates forecasts with the interval score at 90% nominal coverage, charging for width and adding a penalty of 20 times the distance of any miss. In ETF11, Rare replay records a high-VIX score of 0.0736 against FIFO's 0.0872. Its stress coverage is 0.8478.

State composition earns its keep

Adaptive and weighted conformal methods have plenty of precedent. The contribution here lies in controls that test the easy explanations for replay's gain.

Perhaps replay helps simply by activating a different number of residuals. Recent-n tests that explanation: it matches Rare replay's active count using only recent errors and scores 0.0839. Perhaps access to any old data is enough. Hybrid-n uses a single regime-agnostic reservoir of the same size in place of the state reservoirs and scores 0.0855. The paired contrasts against Rare replay are -0.0102 [-0.0213, -0.0019] and -0.0119 [-0.0242, -0.0017].

Oldest-n, an appendix control, selects the oldest stored residuals without regard to state. Its replayed errors have a median age of 3,122 origins, versus 2,784 for Rare. Despite those older errors, its high-VIX score is 0.08509 against 0.07554 for Rare, both under the fixed-alpha variant. Composition beats age.

At recurrent stress entries, the composition difference is large: high-state errors make up 9.57% of FIFO's active set and 34.44% of Rare replay's. The 48 replayed residuals have a median age near 2,914 origins.

For a practitioner, two appendix results deserve more attention than the headline. Adding log VIX to the HAR mean leaves FIFO's ETF11 stress coverage little changed, at 0.7459 versus 0.7576. Feeding lagged VIX into the point forecast therefore does not repair the band. Turning off ACI feedback brings FIFO's stress coverage down to 0.6987, even as aggregate coverage reaches 0.9033. The controller restores about six points of stress coverage, from 0.6987 to 0.7576. A 14-point hole remains.

Replay pays for the improvement. It still undercovers during stress, and its ETF11 intervals widen to 0.0345 from FIFO's 0.0261, about 32% wider. Extra width is inexpensive under this score when stress coverage is near 0.76. Yet the matched controls widen too, and their scores remain 0.0839 and 0.0855 against replay's 0.0736.

The intervals are pointwise. The authors explicitly say they have made no adjustment for multiplicity; the Rare-minus-FIFO upper bounds are -0.0018 and -0.0009. They also acknowledge that both panels use one calendar. Stock50 thus checks cross-sectional transfer over the same dates. Replay alters storage and retrieval together, so "their separate causal contributions are not identified."

The first five days carry the largest gain

An availability mechanism should help most when stress returns, with the advantage fading as FIFO replenishes its stress errors. The results follow that pattern. Across the first five high-state dates in recurrent episodes, Rare minus FIFO is -0.0232 for ETF11 and -0.0202 for Stock50. Later in the episodes, those differences narrow to -0.0052 and -0.0081. FIFO's high-state share rises from 10.37% to 14.71%. At strict-eviction entries, when FIFO contains zero stress residuals, the gaps are -0.0386 and -0.0505.

That last row rests on three entries.

There are 13 stress entries in the calendar, of which 12 are recurrent. Replay wins 10 of 13 ETF11 episodes and 9 of 13 Stock50 episodes. The authors describe this as a timing diagnostic, which is the right description. Dropping any one episode leaves the pooled ETF11 effect between -0.01539 and -0.00994. Defining stress through SPY 22-day realized volatility produces 56 dates, with 43 already classified as VIX-high. Its intervals cross zero, including [-0.0531, 0.0033] for ETF11.

A FIFO weighting scheme leads the appendix

The paper argues that eviction determines which records later weighting can use. Its appendix makes that argument less straightforward. RWC-adapted, its regime-aware weighting method, retains the ordinary FIFO buffer of the last 250 errors. It weights errors using an age half-life of 62.5 origins and a state-distance penalty with gamma of 1.

RWC-adapted achieves a high-VIX score of 0.0636 in ETF11. That beats Rare replay's 0.0736 and the unbounded state-history benchmark's 0.0688. In Stock50, the corresponding figures are 0.0851 against 0.0925 and 0.0878. Stress coverage reaches 0.9009 in ETF11 and 0.9023 in Stock50. The contrast with replay is -0.0100, a point estimate running against the availability explanation.

Selection gives that edge some assistance. The authors chose RWC-adapted from a 16-point grid of half-life and gamma settings. They ranked candidates first by high-VIX interval score on ETF11 validation data from 2009 to 2011. Its point-estimate advantage, 0.0636 vs 0.0736, therefore partly reflects tuning for the metric it subsequently wins. Rare's inverse-frequency allocation was also chosen before evaluation. Rare minus Mondrian is +0.00272 in ETF11, where Mondrian is lower, and -0.00157 in Stock50, where replay is lower. Every interval includes zero.

The appendix concludes: "These results do not identify a uniformly superior state-aware implementation." For RWC-adapted minus Rare, the 20-day interval is [-0.0253, 0.0011], including zero. The authors also say the RWC-adapted versus RWC-MM comparison "changes total calibration mass". In their discussion, they answer that reweighting "can act only on records that survived storage." That is true. The three strict-eviction entries are where this should matter, because RWC has no stress residuals available to upweight.

We did not find RWC results separated by re-entry window. That table would speak most directly to whether storage or weighting is the binding choice. Replay's case would strengthen if RWC scored no better than FIFO on those three entries. The inverse-frequency tilt itself carries little weight in the argument: equal, proportional and inverse-frequency replay all beat recent-only controls at 20/30. With an 18/28 split in ETF11, Rare minus proportional is -0.00011 [-0.00085, 0.00051].

The useful change in practice is to report stress-state coverage for every bounded calibrator. With its controller off, ETF11 combined 0.9033 aggregate coverage with 0.6987 stress coverage. Replay offers one credible repair; the paper also tests Mondrian partitioning and regime-aware weighting. The evidence for each ends at forecast loss.

Our ETF sizing book, 2020 to mid-2024

We implemented one automated version of the idea as a risk-sizing rule. The figures in the strip above are ours, from a backtest covering January 2020 to July 2024. Over January 2020 to July 2024, the book earned 34.59% in total. Sharpe was 0.85, Sortino 1.15, annualised volatility 9.02% and maximum drawdown -14.43%, with Calmar 0.47. A cash-diluted long-only ETF basket producing a 0.85 Sharpe over a period containing two VIX episodes above 30 is unremarkable. It tells us nothing about replay versus FIFO.

We assign each of the 11 ETFs a sleeve of 1/11 times min(1, r_ref divided by its replay upper bound). Here r_ref is the median FIFO upper bound during a 2017 to 2019 validation window. Each sleeve is capped at 9.09%. Capital left unallocated earns nothing in cash. We used reservoir seed 0, the paper's 125/46/31/48 allocation and its 20/30 VIX thresholds.

Several implementation decisions differ from Han and Lin's. Our prices are adjusted; theirs have adjustment disabled. We refit HAR monthly rather than at every target date. Under our daily auction convention, fills occur at the close. Every daily rebalance incurs commissions of $0.004 a share, subject to a minimum of $1.

The paper reports no trading performance. Its performance measures are coverage, width and interval score, none of which is a return. Its ETF11 high-VIX interval score is 0.0736 for replay and 0.0872 for FIFO. Our Sharpe and drawdown describe a different object, leaving no direction in which to compare them.

Our long-only basket moves further into cash as bands widen. Cash partly suppresses the 9.02% volatility, since each sleeve contracts when its band widens; we did not isolate that effect from calibration. Testing the paper would require replay minus a FIFO book over identical days, with attention concentrated on the high-VIX stretches. Our single automated pass provides evidence about our implementation before it provides evidence about theirs.

A prospective comparison would be cleaner: freeze the 46/31/48 reservoirs and RWC at half-life 62.5 and gamma 1, then compare each book with a FIFO book at the next VIX entry at 30 or above. The next VIX>30 entry is the comparison worth waiting for.

Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

1. Validate adjusted ETF OHLC and preceding history. Build Y[t] = log(high[t]/low[t]) before merging VIX.
2. Shift VIX close by one VIX observation, then exact-date join to target dates; classify L < 20, M = [20,30), H >= 30. Do not fill missing data.
3. After raw index 252, fit each ETF's OLS HAR at permitted month-ends using at most 756 completed valid pairs. Freeze coefficients for the following month.
4. Forecast the next target's range from an intercept and lagged 1-, 5- and 22-observation ranges; floor the point forecast at zero.
5. After each target completes, evaluate its previously issued interval, update alpha = clip(alpha + 0.005*(0.10 - violation), 0.001, 0.30), then insert its absolute forecast error.
6. FIFO retains the latest 250 errors. Rare replay retains a 125-error recent queue; route evictions to their stored issuance-state reservoir using independent reproducible reservoir sampling, with L/M/H capacities 46/31/48.
7. Query all recent records plus only the current-state bank. With n active errors, q is the order statistic at min(n, ceil((1-alpha)*(n+1))); interval = [max(0, forecast-q), forecast+q].
8. Freeze r_ref as the pooled median of positive finite FIFO upper bounds during validation. At the evaluation boundary reset alpha to 0.10, preserving memories, counters and RNG state.
9. Set each target weight to (1/11)*min(1, r_ref/upper_bound). Use zero for invalid forecasts/bounds/states or fewer than 20 active records. Hold all unallocated capital as non-interest-bearing cash.
10. Rebalance daily without shorts or borrowing; preserve cash feasibility and skip unavailable execution prices. Mark terminal holdings without forced liquidation.

The specification requires following-open fills, overnight attribution and independently accounted comparison books. The supplied charged-execution record instead identifies daily MOC fills; the reported results therefore use the platform's closing-auction convention rather than establishing the specified next-open implementation. Seed 0 defines the primary investable replay; seeds 0–19 are separate robustness runs, not averaged portfolios.