Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysis

Differenced Random Forest trims Robusta MAE by $0.27/t against no-change forecast

Differencing shields nonlinear models from the 2024 to 2026 price shift; none of the three leaders significantly beats yesterday's close.

2026-11-06 · 4 min read · Machine-Learning Futures Forecasting · ICE Futures Europe Robusta Coffee futures

Reviewing: Fair Walk-Forward Forecasting of ICE Robusta Futures: Comparing Statistical, Machine-Learning, Deep-Learning, and Hybrid Models · Quang Phung Duy, Cao Ha Hai Dang, Thinh Nguyen Huu et al. · Read it on openalex

A differenced Random Forest improves on yesterday's ICE Robusta close by just 0.27 USD per metric ton in mean absolute error. Switching from price levels to first differences cuts the forest's blind-test error by 61.87%, but nearly all that improvement comes against a level-price forest unable to extrapolate: MAE 219.22 versus 83.59. Phung Duy Quang and co-authors do readers a service by putting both comparisons in view.

Inside the horse race

The authors compare 14 specifications on a single daily series: the ICE Futures Europe Robusta benchmark exported by Investing.com. It contains 4,603 observations from January 2008 to January 2026. Two specifications supply the benchmarks, Naive (yesterday's price) and an AIC-selected ARIMA(0,1,1). Four nonlinear learners fit price levels: Random Forest, RBF-SVR, LSTM, GRU. Another four combine ARIMA with recurrent networks. The Zhang version asks a network to forecast the ARIMA residual; the Khashei-Bijari (KB) version also supplies lagged prices. The remaining four apply the same learners to first differences. Each learner receives 60 lags of its target, along with Volume, the broad dollar index, the effective fed funds rate and WTI.

Reconstruction does the heavy lifting. A Delta model, which predicts tomorrow's change, adds that prediction to today's observed price. Every forecast therefore begins at the last close. A level model instead maps MinMax-scaled history to a price, and a tree's output stays within the leaves learned during training. The authors locked specifications using 3,657 pure-training days and 646 validation days ending November 2024. They then refit the models and scored them once, one step ahead, over the final 300 days from November 2024 to January 2026.

At 83.59, Delta-RF has the lowest MAE. KB ARIMA-LSTM leads on RMSE with 111.99. The level learners fall apart: RF records 219.22, SVR 220.75, LSTM 493.67, and GRU 551.02 with an R² of 0.156.

Does the ranking buy anything over yesterday's close?

Very little. Naive records 83.86 and ARIMA 83.87; only 0.28 USD per ton separates the top five models. Against the blind-test mean price of 4,574.63, Delta-RF's advantage over Naive amounts to 0.006%. And the MAE winner loses on RMSE: Delta-RF records 112.62 against Naive's 112.09, taking slightly larger misses than the no-change forecast.

KB ARIMA-LSTM improves on Naive's RMSE by 0.10. Its MAE averages 83.82 across five seeds, with a standard deviation of 0.10. In the DM table, KB ARIMA-LSTM's MAE advantage over Naive is 0.099, equal to the 0.10 standard deviation of its MAE across five seeds.

The Diebold-Mariano tests tell the same story. For Delta-RF versus Naive, the mean loss difference is -0.271, the DM statistic is -0.516 and the raw p is 0.606. Raw p exceeds 0.59 in all eight selected absolute-loss comparisons. Every Holm-adjusted p is 1.000.

The authors handle that result carefully. Failure to reject equal loss supplies no evidence of equivalence, they note, and 300 observations give limited power to detect such small differences. We accept that qualification. A desk still needs a reason to pay KB ARIMA-LSTM's training cost of 28.14 seconds against Naive's zero, and these results supply none. The authors arrive at the same judgement: when transparency and maintenance outweigh nominal rank, the costs favor Naive, ARIMA or Delta-RF.

Longer horizons offer no rescue. Across 281 rolling origins, the Delta-RF minus Naive MAE gap ranges from -0.37 to +0.69. Delta-RF trails Naive only at h=5.

The 2025 price shift broke the level models

The blind-test mean price reaches 4,574.63, compared with 1,943.31 in training. For price, the KS statistic is 0.955 and the standardized Wasserstein distance is 4.274. Fifty-six test prices fall outside the full training range. The level forest has learned no leaf for those prices.

Delta models escape that problem. The permutation test then measures what Delta-RF contributes after escaping it: shuffling all 60 lags of price changes increases MAE by 0.755 (SD 0.356). Those lags nevertheless account for 61.3% of impurity importance. The dollar index contributes -0.002 and Volume -0.067; EFFR adds 0.041 and crude 0.025. The macro regressors are decoration.

The abstract makes the same argument. The authors write that differencing does not establish economically or statistically meaningful predictive information beyond the most recently observed price. They also place the protection against a shift in the price range where we would: reconstruction brings forward the last observed price. Relative to Naive, that last price supplies the protection. No Delta model significantly beats Naive (Delta-RF raw p 0.606).

For LSTM, the Delta switch reduces MAE by 83.02%; for GRU, the reduction is 83.99%. Those comparisons need a qualification. Early stopping fixed the level LSTM and GRU at one epoch each, whereas Delta-LSTM trained for 36 and Delta-GRU for 2. We attribute part of the LSTM gap to tuning. The GRU reduction compares one epoch with two, leaving little weight behind an under-training explanation there. RF's 61.87% reduction, obtained with symmetric 12-candidate grids, makes the cleaner comparison.

Discipline worth keeping

The authors' testing discipline travels better than the forecast ranking. Every choice is locked before the blind test, and both Naive and ARIMA remain in the comparison. Holm correction covers 16 prespecified comparisons. The prediction vectors are available in a public repository, allowing anyone to rerun the DM tests.

The ranking supports no P&L claim. As the authors disclose, the export has no delivery-month identifiers, leaving roll discontinuities inseparable from price moves. Their evaluation measures point-forecast accuracy only; transaction-cost-aware decisions remain future work. The authors themselves judge a 0.27-dollar edge too small to cover spreads and basis risk.

We could not rerun the study. Our futures coverage excludes Robusta, and our macro feed lacks both the broad dollar index and the WTI spot series. The sample also begins in 2008, before our price coverage.

A DM rejection against Naive on a second, roll-mapped contract would move us.