Historical expected shortfall calculated from 250 rolling ten-day returns sets risk limits too low, according to this paper. The evidence supports that direction. How far the estimate falls short on real books is less certain, a weakness the authors partly acknowledge themselves.
The construction in Aichele, Cialenco, Jelito and Pitera
The paper works within the FRTB internal model: ES at 2.5%, a one-year window of n = 250 scenarios, and a ten-day liquidity horizon. Banks construct ten-day P&Ls from daily observations, advancing the window a day at a time. Adjacent scenarios share nine of their ten days. Yet the estimators used on this sample were derived for independent draws. Historical simulation is used by 71% of banks disclosing their VaR approach, according to the EBA. This mismatch is therefore the industry default.
From an EBA catalogue, the authors select six order-statistic ES estimators and treat them as coherent risk estimators. These are rules that satisfy the coherence axioms on the sample itself. Their earlier work establishes that the comonotonic, law-invariant rules are precisely weighted averages of ordered P&Ls, with non-negative decreasing weights summing to one. At n = 250 and 2.5%, these six estimators put their entire weight on the six or seven worst outcomes. Most of the difference lies in how much weight the worst loss receives: 0.160 for the plain empirical plug-in (#1), rising to 0.295 for the Pareto-extrapolated version (#6).
The simulations use 10^6 Monte Carlo samples per model, both with overlap and without it. They cover Normal, Student-t5, six NIG laws, two GPDs and four GARCH(1,1) specifications. The market tests use twelve risk factors from 2010 to 2025 and 99 Fama-French size/book-to-market portfolios.
The market evidence rests on the safe confidence threshold, CT. To calculate it, add the estimated reserve to the next realized P&L and find the ES level at which this secured position first becomes acceptable. A correctly estimated ES 2.5% reserve produces CT near 2.5%. CT describes the tail of the secured position. It measures something different from a VaR exception count.
250 overlapping Gaussian scenarios have the precision of about 57 independent ones
The Gaussian experiment supplies the paper's cleanest comparison. Summing ten Gaussian days leaves the law Gaussian. The target distribution keeps its shape, so dependence is the only difference between the designs. For estimator #1, the variance from 250 overlapping sums matches that from an independent sample of n ≈ 57. Precision falls by more than a factor of four.
The estimate also shifts downward. On the Normal, overlap takes #1's statistical bias from -1.4% to -6.7% and CT from 3.0% to 5.5%.
Heavy tails deepen the problem. With overlap, median bias across the six estimators reaches about -19% on GPD with ξ = 1/3. Risk bias is the residual ES of the secured position divided by the true ES; there it ranges from -38.6% to -40.1%. Volatility clustering has the same directional effect even without overlap. On the fitted GARCH-norm model, #1 has bias of -10.7%, worse than the -6.7% caused by overlap alone on the Normal.
The authors identify the limitation. Outside the Gaussian experiment, the ten-day law is closer to Gaussian than the daily law. Those comparisons therefore combine dependence with a change in the target distribution. The Gaussian result establishes the sign despite that objection. It leaves the size of the effect on fat-tailed P&L unresolved.
Can heavier weighting of the worst loss repair the bias?
Only partly. The abstract says conservative weights reduce underestimation without eliminating it. The conclusion adds that #5 and #6 remain in the safe zone on static models without overlap. Once scenarios overlap, the improvement falls below a point of CT.
The direct comparison of #1 with #6 shows the trade. Without overlap on the Normal, #6 takes bias from -1.4% to +2.8% and CT from 3.0% to 2.4%. RMSE increases from 8.6% to 9.9%. Safety comes at a cost to fit.
Overlap reduces that trade to a shift in level. On the Normal, RMSE changes little, from 18.9% to 18.6%. Bias improves from -6.7% to -3.9%, while CT falls from 5.5% to 4.8%. For GPD ξ = 1/3, bias moves from -20.6% to -18.7% and CT from 9.3% to 8.8%. The fitted GARCH-t model sees CT fall from 9.8% to 9.1%. Each improvement is under a point of CT against a 2.5% target.
A complication weakens the authors' case slightly. They rescale the weights of #3, #5 and #6 to sum to one so that the estimators are coherent. This reduces #6 by 7.4%. They disclose the adjustment and explicitly say their figures cannot be compared directly with the EBA's. Since the mean scales linearly, leaving #6 unrescaled would raise its mean estimate by about 8%. Our arithmetic takes the overlapping Normal bias from -3.9% to roughly +3.8%. For GPD ξ = 1/3, it takes -18.7% only to about -12%.
On that arithmetic, rescaling alone explains the remaining mean shortfall of #6 on the overlapping Normal. None of the six weightings closes the gap for the heavy tails seen in bank P&L. The authors also doubt that re-weighting estimators inherited from the i.i.d. setting would do so. They report median Hill tail shapes of 0.328 on ECB trading-book P&L.
The ES/VaR ratio clears the check as realized CT doubles
Model validation teams should act on this result. Across market risk factors, the cross-series median of the mean ES 2.5% / VaR 1% ratio ranges from 0.953 to 1.022 on daily data. With overlap, it ranges from 0.958 to 0.995. Every series' mean ratio stays above the 0.9 floor under both schemes. At the ratio's 2.5% quantile, however, #1 breaches the floor in 12 of 12 series on daily data and in 8 under overlap. Dispersion actually narrows: #1's median standard deviation drops from 0.065 to 0.036.
Realized CT tells a different story on the same data. Its median rises from 4.0% (#1) and 3.2% (#6) on daily returns to 8.0% and 7.2% with overlap. For S&P 500, #1 goes from 4.3% to 9.2%. The Fama-French portfolios cluster at 3% to 5% daily, with medians of 8% to 9% under overlap. Diversifying across 99 long equity books buys nothing here. The ratio cancels out deterioration affecting its numerator and denominator through the same channel.
The authors argue that the regulatory 1-day backtest may miss the problem. The 1-day VaR estimator records a median exception rate of 1.1%, against a nominal 1%. This accurately describes one-day quantiles. It says nothing about coverage of the ten-day tail.
The market CT figures carry two cautions. Each is a single point estimate for a series, based on roughly 1,700 to 3,900 secured observations. Those observations also overlap under the ten-day scheme, and we did not find confidence intervals for them. The two short-rate series have Hill estimates of 0.97 and 0.48 and reach CT of 14.0% to 20.5% with overlap. The authors retain them as stress illustrations. A median across twelve series is less exposed to them than a mean would be. Even so, the per-series gaps of 1 to 6 points on the ten regular series need error bars.
An adaptation to SPY sizing
We could not trade the paper's panel. The twelve risk factors and 99 Fama-French portfolios were unavailable to us, and our run does not reproduce them. We implemented only two of the paper's six weightings: #1 and the rescaled #6 with ξ = 1/3. For those two rows, our weights match the paper's table. We used SPY alone, adapting the estimation mechanism to a single equity ETF. This does not test the paper's claims.
The overlay allocates min(1, 0.05 / ES) to SPY and holds the balance in cash. It estimates ten-session ES from 250 trailing scenarios, using closes through the previous session, and rebalances at the close. Our window covers January 2020 to July 2024. We charge commissions of $0.004 a share, subject to a $1 minimum. Crossing the two estimators with overlapping and non-overlapping scenarios produces four arms. Only one of the four traded.
From January 2020 to July 2024, the traded arm returned 32.80% in total, with a Sharpe of 0.62 and a Sortino of 0.77. Maximum drawdown was -20.85%, annualised volatility was 11.77%, and Calmar was 0.31. The result is unremarkable for a book capped at 100% SPY. Since only one of the four arms traded, it tells us nothing about overlap versus non-overlap.
Our own design choices shape these figures more than the paper does. The long-only book has a 100% SPY cap, bounding its beta and volatility by SPY's. The non-overlapping arm also requires 2,500 daily returns, about ten years, compared with 259 for the overlapping arm. Dependence is therefore mixed with lookback length. This resembles the confound the authors acknowledge when the target law changes. The paper is a capital study and reports no trading performance, leaving no published performance figure against which to compare ours.
A version worth trusting would include a matched-horizon control: equal counts over different spans, or equal spans with different counts, reported both ways. It would also include the 60-session block-bootstrap intervals we specified.
The paper gets the direction right.
To persuade me on magnitude, it would need realized CT with confidence intervals on a matched-horizon market sample. We did not find that in the paper. Its comparison of daily data with overlapping ten-day data deliberately mirrors practice.
Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.