A single empirical quantile, taken from one vector sum of daily high-low-close bars, gets remarkably close to nominal coverage. Across 500 random portfolios trading 49 correlated futures, the median 99% daily VaR breach rate misses the target by less than a tenth of a percentage point. The calculation needs no covariance matrix or copula. Dispersion across individual portfolios is considerably larger: 68% of the 500 fall within 1.0 +/- 0.3%, while 95% fall within 1.0 +/- 0.5%, with whiskers at the 2.5% and 97.5% quantiles. The presented evidence supports that result.
Our figures require a disclosure first. The paper trades 49 futures across specific front and back-month expiries that we cannot trade, so our run substitutes a diversified US equity book. It tests calibration of the risk estimator on that substituted universe. It does not reproduce or transfer the paper's futures-universe results, and it is not a replication of the paper.
The paper rests on one substitution, and the mechanics are easiest to judge once that substitution is clear.
Past shocks, repriced at today's volatility
Begin with the current portfolio. Each of K instruments has an exposure expo_k, defined as market value over AUM. Sun estimates recent volatility for each instrument from its daily true range percentage. The measure takes the day's high-low span, extends it to include the prior close, divides by that prior close, and calculates the root-mean-square over the last 14 days. Sun calls the result sigma14.
Now choose a past day j. Instrument k recorded return r_kj on that day, when its trailing 14-day volatility was sigma14 measured through j-1. Dividing the return by that volatility produces a unitless shock, expressed in units of the instrument's volatility for that day. Repeat this across all 49 instruments. The result is one vector of 49 simultaneous standardized shocks, preserving the cross-asset relationships that actually occurred.
Each element is then multiplied by current exposure and current volatility. Sum the vector, and it becomes the return the current portfolio would have earned if day j's standardized shocks arrived tomorrow under current volatility. Repeating the exercise across the look-back window creates an empirical distribution. Its 1% quantile is the 99% VaR.
This construction retains the features that conventional modelling often removes. Observed joint crash days remain intact. So does left-tail skew. The positive skew of VIX futures against equities also remains, without requiring anyone to specify it. Sun treats each historical day as a Monte Carlo trial. In his framing, the quantile integrates above and below a K-1 dimensional surface oriented by the current exposure-times-volatility vector. "Here, we let past data speak for itself with no assumptions or approximations."
There are only two settings to choose: the volatility length n and the historical-window length. The test universe contains 49 liquid futures, with front and back month treated as separate instruments. It spans VIX, five equity indices, Euribor, five government bond contracts, energy, metals, livestock, grains and softs. Sun constructs 500 portfolios with daily random exposures given by (1/1000) x (1/sigma252) x Uniform(0.3, 1.0) x RandomSign. Each VaR estimate for day i uses information only through i-1. He then records every day on which the realized portfolio return falls below that estimate.
Where does dimensionality still hurt?
The computational observation is sound and unsurprising. Monte Carlo integration error scales as 1/sqrt(N) regardless of dimension, while evaluating a portfolio-loss quantile requires a vector sum and a sort. Sun argues that total dimensionality does not affect the method's performance. His experiment covers 49 dimensions. Covariance inversion and kernel smoothing really do deteriorate as dimension rises.
The statistical problem remains, as Sun acknowledges.
A 1260-day window contains roughly 13 observations below the 1% surface. A 2520-day window contains roughly 25. Tomorrow's tail estimate therefore depends on about a dozen historical rows, and our reading of the arithmetic is that the same rows continue to determine the quantile as the window advances. Sun suggests two responses. The first is to estimate 95% or 90% VaR instead of 99%, putting five times as many observations below the integration surface. The second is to back-fill short histories through simulation, an idea he proposes without testing.
He states the difficult case directly. With 200 days of data, "there simply does not exist enough data to say anything statistically significant about an event that is expected to have occurred only 2 times in the past." That concession is fair. Sun has shifted the difficulty from model specification to data sufficiency, and data sufficiency remains the binding constraint.
500 random books across nine settings
Across the nine parameter combinations, median breach rates range from 0.90% to 1.09% against a 1.0% target. The table entry we prefer uses 14-day volatility and a 1260-day window. Its median is 0.97%, with the table's narrowest 95% interval, from 0.73% to 1.30%.
Sun does not designate a best configuration. Judged solely by the median, 30-day volatility over 2520 days comes closer, at 0.99% versus 1.00%. Longer histories make the estimate more conservative. With 14-day volatility and 2520 days, the median falls to 0.90%, and the 95% interval runs from 0.59% to 1.29%. More history therefore adds caution rather than accuracy. Extending the volatility window moves the estimate in the opposite direction, reaching a 1.09% median at 45 days and 1260. The table's widest cross-portfolio interval is 0.64% to 1.45%, produced by 45-day volatility on all history.
We could not find the backtest's start and end dates anywhere in the paper. The available timing information gives only the binding constraints: VIX futures begin in 2004, back-month volume begins in 2006, and the first estimate requires 1260 to 2520 days of warm-up. This omission affects how the dispersion should be read.
Assume each portfolio supplies on the order of 3,000 daily observations. The binomial standard error for a 1% rate is then about 0.18%. Much of the reported cross-portfolio spread would fall within ordinary counting noise: 68% of portfolios are within 1.0 +/- 0.3%, 95% are within 1.0 +/- 0.5%, and the whiskers sit at the 2.5% and 97.5% quantiles. Without the number of observations, readers cannot distinguish variation in model risk across portfolios from the arithmetic of rare-event counts.
The random exposures are intentional, and Sun explains why. Without inverse-volatility scaling, natural gas and Treasuries cannot share one test because "a few instruments always dominating" would undermine the purpose of a 49-dimensional experiment. He also admits that "no one will trade a randomly positioned portfolio." Daily re-randomized, volatility-normalized long/short positions spanning nine asset classes put heavy pressure on the aggregation calculation. They leave persistence largely untested, including a book that carries the same crowded exposure for six weeks while the estimator's tail observations age.
Crisis performance comes from fast adaptation
Under 14-day volatility, monthly average breach rates top out at 2.5% to 3.0% during the worst months, March 2020 and the 2018 volatility blow-up. The same months reach 6% with 30-day volatility and 8% with 45-day volatility. Introduce a one-day information delay, so day i is sized from data through i-2, and the medians remain inside 1.0 +/- 0.1%. For the 14-day volatility configuration, the delay raises crisis peaks from 2.5%-3.0% to 3.0%-3.5%.
Fast adaptation carries these results. N appears twice, once in current volatility and again in every historical normalizer. The contrast between 3% and 8% therefore compares two filter speeds rather than a filter with retained history. Sun openly says that the estimator ignores autocorrelation between past days j. His response is empirical: monthly breach rates stay near 1% when n remains short.
Sustained breaches would require volatility to increase geometrically week after week, 4 x 4 x 4 = 64, eventually exceeding 100% daily volatility. The arithmetic holds. The supporting evidence is weaker than the table's size implies because all 500 portfolios trade the same 49 instruments over the same calendar. March 2020 supplies one shared shock, rather than 500 independent observations. Sun notes that 3% of 22 days equals 0.66 breach days. That framing cuts both ways: at monthly resolution, a single observation cannot separate independence from mild clustering.
A smaller evidentiary gap remains. The text says close-to-close RMS performs similarly to true range, yet we did not find a table or figure supporting that comparison.
CVaR appears in the abstract, then disappears
The abstract promotes a "non-parametric value-at-risk (VaR+CVaR) algorithm." The empirical test covers only the 99% daily VaR. Sun says directly that CVaR backtesting remains a future exercise, together with weekly and monthly horizons.
He gives the conceptual answer immediately. The algorithm estimates the full portfolio return PDF, allowing VaR or CVaR at any confidence level to be taken from it, with expected shortfall defined as the expected value of the PDF's tail. Accuracy of the resulting shortfall estimate remains a separate empirical question. Breach counts do not answer it. A model may achieve 1.0% coverage while consistently understating the average exceedance.
We also found no Kupiec unconditional coverage test or Christoffersen independence test. The paper offers no comparison with the natural alternatives, plain historical simulation and GARCH-filtered historical simulation, which applies the same volatility-standardize-and-rescale idea through a parametric filter. This is the omission I would most like to see addressed. The relevant comparison is whether the true-range 14-day filter, combined with untouched cross-sectional rows, beats a GARCH filter applied to the same data.
Our run uses a different universe
Because we could not trade the paper's instruments, we replaced them with an equal-weight long-only portfolio of the top 20 US large caps. We estimated risk from daily bars for those stocks. The overlay follows Sun's Equation 11 delayed variant, using information through i-2, a 2520-day scenario window and 14-day true-range volatility. Positions are scaled by min(1, 0.01 / VaR loss) against a 1% daily loss budget, with all remaining capital held in cash at zero. Sun instead sizes the test portfolios using inverse 252-day volatility and never adjusts exposure to meet a VaR budget. The 1% budget and multiplier are our additions.
From 2020-01-01 to 2024-07-01, the run earned 17.59% in total. Sharpe was 1.58, realized volatility was 2.48%, beta to SPY was 0.03, and maximum drawdown reached -3.42%. The strategy made 6,033 daily rebalances. We charged no commission or slippage, so the reported 17.59% is gross.
The volatility and beta matter more than the Sharpe. With 2.48% volatility, 0.03 beta and a -3.42% maximum drawdown, the portfolio spent almost all its time in cash. A long-only mega-cap book has much higher cross-sectional correlation than a randomly signed 49-contract futures portfolio. Its empirical 1% tail consequently stays above a 1% daily budget almost all the time, forcing the multiplier sharply lower. The 1.58 Sharpe reflects our invented budget parameter more than the estimator itself.
Sun reports no return, Sharpe or P&L result of any kind. The paper's headline figure is a median 99% VaR breach rate of 1.0 +/- 0.1% across 500 portfolios. Our run generated no breach-rate series, leaving no like-for-like figure for comparison. The measurements answer different questions. Sun reports calibration on multi-asset futures, while our figure is the P&L from a cash overlay on US equities.
We cannot fully account for the shape of our result from the information available. One unresolved issue is the multiplier's sign convention. Interpreted as a signed quantile, the rule keeps exposure at 1. Under a positive-loss interpretation, it halves exposure, and that choice changes every figure above. Our single automated pass provides evidence about our implementation, rather than a verdict on Sun's calibration claim.
We have encountered this pattern before in another form: a carefully constructed estimator evaluated against a quantity that nobody holds in practice (the rotation premium no traded instrument spans).
I would use this tomorrow as a baseline risk number. It is cheap, requires no asset-class judgement, and operates on positions rather than strategy P&L. Sun's argument that a risk model fed realized returns rewards past selection skill by treating it as low risk is the strongest page in the paper. Median breach rates from 0.90% to 1.09% across nine settings, measured against a 1.0% target, amount to a genuine calibration result.
Expected shortfall would change my view of the method as a headline risk measure. I want exceedance magnitudes after each breach, tested on a portfolio with persistent exposures and compared with a GARCH-filtered benchmark on the same panel. Until those results exist, the demonstrated claim remains a breach count, produced from portfolios that the author agrees nobody will hold.
Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.