Calling next-day Bitcoin direction correctly 76.24% of the time out of sample, with a frictionless Sharpe of 6.75, deserves more scrutiny than Shokeralla's headline comparison of forecasting methods. The title concerns machine learning's limits. Beneath it, the table gives OLS a 76.24% hit rate on daily BTC, compared with 47.53% for a naive random walk. The forecasting result is the larger claim. It needs the closer audit.
The forecast inside the risk study
Shokeralla uses daily Yahoo Finance closes for seven assets: BTC, ETH, the S&P 500, NASDAQ, gold futures, WTI and the dollar index. The sample runs from January 2017 to May 2026, with roughly 2,320 observations per series. Risk description occupies most of the paper. EGARCH leads on AIC for five of the seven assets; asymmetry is significant for the S&P 500 (alpha1 = -0.182) and NASDAQ (-0.156), though not for BTC (p = 0.943). PELT, a penalised search for changepoints, identifies zero mean breaks across all seven series and 5 to 11 variance breaks in each. A peaks-over-threshold model of the tails puts ETH and BTC highest for 95% expected shortfall, at 13.182% and 10.005%. Gold fares worst in the VaR calibration backtest: exceedances reach 8.90% at the 95% level.
The trading forecast covers BTC alone. Six models predict its next daily return: naive, AR(1), Elastic Net, OLS, random forest and XGBoost. Inputs include BTC return lags of 1, 2, 3, 5 and 10 days, plus a five-day rolling mean and standard deviation of BTC returns. The other six assets enter through one-day lags. The first 70% of the sample supplies the training data, and the stated test window runs from September 2024 to May 2026. The proposed explanation is short-horizon time-series momentum. Recent BTC returns persist in a linear relationship that OLS captures, leaving trees nothing to add. Positions follow the forecast's direction each day.
For XGBoost against OLS, the headline test reports a DM of -0.403, p = 0.687. OLS records both the lower RMSE, 0.02034 against 0.02057, and the higher hit rate, 76.24% against 72.47%. Random forest performs significantly worse than either. Its DM against OLS is -2.653 (p = 0.008), and against XGBoost it is 2.197 (p = 0.029).
Can the reported dependence explain half the forecast error?
The naive random-walk forecast has an RMSE of 0.04130 and gets direction right 47.53% of the time. OLS cuts that error in half. Using the table's figures, its mean squared error is about a quarter of the naive benchmark's: ((0.02034/0.04130)^2 ≈ 0.24). Yet the paper gives BTC a full-sample Ljung-Box p of 0.025 and a daily SD of 4.185%. BTC's lags clear Ljung-Box at p = 0.025; the S&P 500 and NASDAQ clear it at p < 0.001. The reported serial dependence is too slight to explain halving forecast error in a series with this volatility.
The ML verdict depends on one split, and the cited yearly checks remain within that test window.
A five-day mean gets 2.65 times the next feature's SHAP weight
The leading feature is roll_mean_5, the five-day mean BTC return. Its mean |SHAP| is 0.02339, about 2.65 times the 0.00883 for lag2. The paper interprets this as linear momentum that OLS can extract as readily as XGBoost. SHAP describes how much a feature influences XGBoost's output; it says nothing about linearity or cause.
If the rolling window spans t-1 to t-5, the feature is nearly a linear combination of lags 1, 2, 3 and 5, already included among the inputs. In that case, SHAP distributes one signal across overlapping features. The ranking also looks odd for momentum: lag1 (0.00736) falls below lag2 and lag3. The top feature shifts XGBoost's predictions by 0.023 on average, compared with an RMSE of 0.02057 for XGBoost itself.
Where does roll_mean_5 end?
Rolling features, the paper says, "use only past observations up to the forecast date." That wording leaves the decisive question open: does the window end at t-1 or t? With day t's return inside roll_mean_5, the target would account for a fifth of the highest-ranked feature. Every number above would then follow mechanically.
The paper presents three leakage checks: residual Ljung-Box (p = 0.237), yearly hit rates of 70.7%, 76.8% and 79.6%, and a permutation test. The author treats these as settling the leakage question, describing the permutation result as "indicating no leakage". Residual whiteness and consistent yearly hit rates would not expose a feature containing the target. Permuting the features' time order while holding the target fixed takes accuracy down to about 50%. That shuffle breaks alignment for a leaked feature just as it does for a genuine signal, so it cannot distinguish them.
The observation counts also fail to reconcile. Thirty percent of about 2,320 days comes to about 700, whereas the evaluation window has n of roughly 425. BTC trades seven days a week. We did not find an explanation of how weekend BTC returns line up with equity trading days. The note to Table 4 says negative DM favours Model 2, yet the OLS-versus-random-forest result (-2.653) is interpreted as an OLS win. The text gives Elastic Net and AR(1) p-values that do not appear in Table 4.
Shokeralla acknowledges inflation in the Sharpe figures. The simulations are described as "upper-bound estimates of the economic potential of the statistical advantage", and the paper says BTC momentum was mostly favourable during the window. Table 6 supplies the defence. Charging 1.00% per trade still leaves OLS with an annualised return of 0.78, a Sharpe of 2.55 and a drawdown of -0.07. XGBoost returns 0.76 with a Sharpe of 2.48. That haircut addresses friction. But execution assumes daily closes, and a signal that observes its own fill close survives the paper's 1.00% charge.
Rebuilding with BTC's own history
We cannot trade the paper's seven-asset specification. Our run uses BTC alone, with rolling windows ending at t-1. The signal is fixed at the prior close, the position is held over day t, and the benchmarks are AR(1) and a zero forecast. This BTC-only run removes features from the paper's model, so it cannot reproduce the seven-asset analysis. It retains the inputs underlying the top four SHAP features, all drawn from BTC's own history. If the momentum explanation holds, a strictly lagged rebuild should preserve most of the 76.24% hit rate.
On the ML comparison, the abstract gets the wording right: XGBoost does not statistically outperform OLS. The conclusion extends that finding across subperiods and validation schemes. Table 4 reports one DM for each model pair, however, and no yearly DM statistic appears. Supplementary runs on ETH and the S&P 500 report XGBoost-versus-OLS DM p-values of about 0.70 and 0.65, without further figures.
A strictly lagged rebuild with a hit rate anywhere near 76% would change my view of the whole paper. Until then, I would carry away the variance-break finding: zero mean breaks and 5 to 11 variance breaks per asset. The gold-VaR result needs its caveat alongside it. The paper never specifies whether that backtest is in-sample or out-of-sample, and it describes the backtested model as historical VaR while Table 3 reports EVT VaR.