The paper earns its controller claim on the training set. The Bitcoin backtest shows that the learner runs, while the trading case carries less weight. The authors say the evaluation "is confined to one asset with tabular states" and acknowledge that returns remain below buy-and-hold. Their claim concerns improved risk-adjusted performance rather than raw returns.
The fixed-budget problem
The estimator comes from Huang and Haskell (2021). They replace the conditional expectation in the Bellman recursion with a coherent risk functional. For CVaR, the functional takes the optimized-certainty-equivalent form: minimize over y of y plus E[(X - y)+]/(1 - alpha). The minimizing y is a Value-at-Risk threshold.
This leaves the algorithm with two coupled recursions. An inner stochastic-approximation loop tracks y in every state-action cell. The outer loop updates the Q-table from a sampled CVaR target constructed with that y. Convergence theory supplies asymptotic step-size schedules, but offers no allocation rule for a fixed transition budget split between the loops or among cells visited at different rates.
The paper is strongest when it explains the amplification. If y falls below the sampled target x, the CVaR target becomes (x - alpha·y)/(1 - alpha). With alpha = 0.8, the expression is 5x - 4y. Any error in an undertracked VaR estimate reaches the Q-update multiplied by four. Each Q-update also shifts the threshold pursued by the inner loop, allowing both errors to reinforce one another.
The proposed repair changes only the controller through six cumulative adjustments. Inner step size decays according to each cell's visit count instead of a global counter. The algorithm then multiplies that decay by the outer visit count raised to the Q-update's own exponent, slowing the inner loop as its target slows. During the first 5% of the budget, a brief correction moves y toward the clipped sampled target.
Action selection also has two phases. Until 60% of the budget has been used, the controller chooses the least-visited action. It then switches to epsilon-greedy selection with greedy probability 0.90. The outer update uses a suffix average of the inner CVaR targets, and its window expands from a tenth of the visits to all of them. Three constants are calibrated from observables, including the inner-loop depth L. The cube-root rule for L comes from Gordy and Juneja's nested-simulation allocation, although the paper explicitly declines to describe it as optimal here.
The test bed contains 3,059 daily BTCUSDT closes aligned with the Alternative.me Crypto Fear and Greed Index. The stated period runs from 8 February 2018 to 28 June 2026. A chronological 70/30 split yields 2,140 training transitions and 918 test days. Sentiment, seven-day sentiment momentum and recent return each have three bins, producing 27 states. The action set contains six exposures from -1 to +1. Training loss equals minus 100 times exposure times next-day return. The budget covers 856,000 inner transitions, 400 epochs and 20 seeds. Scheme 0, the fixed baseline, hand-sets inner depth L = 80 and inner risk scale h_y = 10.
The main residual figures are large improvements. MeanBEQ declines from 1.2202 to 0.1854, while MeanBEV falls from 1.1624 to 0.0535. MaxBEQ drops from 4.6889 to 1.3302 and MaxBEV from 1.9789 to 0.1959. The maxima carry more information than the averages because a greedy policy can locate the single cell where a residual remains large.
The 85% reduction measures convergence toward the learner's own fixed point on its training data.
What actually drives the result?
The cumulative ablation makes the six-part design look much less unified. Per-cell inner decay alone performs considerably worse. MeanBEQ rises to 3.5297 and MeanBEV to 3.3499, compared with the baseline's 1.2202 and 1.1624. The effective decay weakens, leaving late steps too large.
Outer-rate-matched decay produces the major reversal, cutting MeanBEQ to 0.6122 and MeanBEV to 0.1211. The exploration rule delivers the next-largest change: coverage-first-then-greedy lowers MeanBEQ from 0.6080 to 0.2369. Together, the final two mechanisms move it from 0.2369 to 0.1854. Calibration improves sensitivity rather than the level, narrowing the MeanBEQ range across alpha in [0.50, 0.90] from 0.1910-0.6702 to 0.1663-0.4005.
I read the two largest individual reductions as coming from timescale synchronization and sample coverage. The Scheme 1 to Scheme 2 rows, where MeanBEQ moves from 3.5297 to 0.6122, also show that the first change pays only after the second arrives. The authors give the table another interpretation: they write that the cumulative ablation confirms no single mechanism dominates.
At the terminal checkpoint, the early y correction barely changes the result, moving it from 0.6122 to 0.6080. Under stress, however, it keeps the recursion alive. At inner risk scales h_y of 0.03 and 0.04, all 20 Scheme 2 seeds diverge. Scheme 3 completes at every tested scale and keeps early MeanBEQ between 0.89 and 0.91. For a fragile recursion, that is a genuine engineering result.
One path carries the trading claim
Across the 918 test days, Scheme 6 records an out-of-sample Sharpe of 0.9281, a 6.46% maximum drawdown and 23.61% cumulative return. The figures include 5bp of one-way turnover cost and are averaged over 20 seeds. Scheme 0 records 0.5628 and 17.77%. The paper reports no test statistic for the Sharpe difference. It describes the dispersion directly: "These standard deviations measure variation due to training randomness on the same market path." In my judgement, a useful test statistic is unavailable anyway because all 20 policies were evaluated on the same 918 days.
The abstract concedes that buy-and-hold earns the higher cumulative return, 35.43% against 23.61%. Its answer is far lower volatility, 9.57% against 47.93%, alongside lower drawdown and CVaR loss. Drawdown provides the stronger defence: buy-and-hold reaches 52.21%, compared with 6.46% for Scheme 6. Volatility by itself settles little because Sharpe is leverage-invariant. The paper's table gives buy-and-hold and a static 20% BTC position the same Sharpe of 0.4902.
The static baseline is therefore the harder comparison. Against fixed w = 0.2, Scheme 6 produces 23.61% versus 11.25% cumulative return and 6.46% versus 12.61% drawdown. Annualized volatility is essentially identical, at 0.0957 against 0.0959. On this path, the timing works.
One reported zero remains awkward. Across 20 seeds, annualized volatility is 0.0957 with a standard deviation of 0.0000, matching the static small-exposure policy to two decimals. The learned policy changes exposure enough to double the return without shifting realized volatility away from a constant.
Costs enter after training
Transaction costs never enter the tabular training loss. The paper explains that they depend on the previous position, which lies outside the 27-state Markov description. Costs appear only during evaluation. A policy averaging 0.1740 of daily turnover was therefore never trained to conserve turnover.
Using the paper's figures, 0.1740 × 365 × 0.0005 gives about 3.2% of annualized drag against an 8.79% annualized return. Under 10bp all-in, which is my assumption and not theirs, the same arithmetic removes roughly another 3.2% and leaves annualized return near 5.6%. The controller improves, while the strategy remains exposed to a cost slope absent from its objective. We have documented this failure mode before in trend, where the entire result depended on the location of the cost threshold (/articles/trend-following-as-a-poisson-kernel-read-of-the-spectrum).
Our rebuild was weaker
We ran one automated implementation from the paper's description. It differs from the paper's strategy, and every figure here is ours. Over a 2019-01-01 to 2025-06-30 window with the last 30% held out, our long-or-cash BTC policy returned 16.62% in total at a Sharpe of 0.34. Maximum drawdown was 10.44%, Calmar was 0.16, and the policy made nine trades. Realized volatility came to 5.16%.
On its own 918-day test split, the paper's Scheme 6 reports a 0.9281 Sharpe and 6.46% drawdown. Our Sharpe is about 2.7 times lower and our drawdown 1.6 times worse. Those figures describe different experiments.
The differences are concrete. Our actions are cash or 100% long, preventing the finely graded low exposure behind the paper's 9.57% volatility. With nine trades and 5.16% realized volatility, our policy spent most of its time in cash instead of carrying full market risk. Our state grid uses five features divided into three bins, giving 243 states across two actions, and omits sentiment momentum entirely. The resulting cells receive roughly a dozen inner samples, with around 9 observations per state. Only three or four lie in the alpha = 0.60 tail. We also substituted derived crypto-news sentiment for the Fear and Greed Index.
Two accounting choices mechanically depress our result. We annualized with 252 days, whereas the paper uses 365, and our return series contains 2,373 rows over a window with 1,637 BTC trading days. Correcting both would raise 0.34 to roughly 0.5 before touching any economic difference. Nine trades also provide far too little evidence for comparison with a daily-rebalanced path. This single automated run bears first on our implementation and does not amount to a verdict on the authors' work. We cannot fully explain the remaining gap after sizing the differences available to us.
The paper identifies both gaps that matter most to me. Its closing section says the controller "lacks convergence guarantees for the coupled adaptive recursions, and the evaluation is confined to one asset with tabular states", then proposes multi-asset and function-approximation settings as future work. My requests follow directly: run the same controller on a second asset, or recompute the Bellman residual comparison on the held-out split. Either would change my view. The controller appears to repair a fragile two-timescale recursion. The Bitcoin evidence still comes from one regime, one path and an objective that never sees trading costs.