Chronos-2 cannot beat zero forecasts for five Indonesian bank stocks

Zero-shot Chronos-2 forecasts for five Indonesian bank stocks are indistinguishable from a zero daily return on the paper's own tests. The 90% intervals are too narrow for risk sizing, too. Hozairi, Anwari and Alim acknowledge much of this, which deserves credit. The horizon comparison needs care: h=20 pools 860 forecasts per stock, against 43 at h=1.

The setup

Chronos-2 takes historical observations and produces forecast quantiles without training on the target series. It is a pretrained time-series foundation model. Its multivariate mode reads several series together, allowing information from one to inform another's forecast. That gives the paper a straightforward trading hypothesis. Indonesian bank returns have pairwise correlations ranging from 0.432 (BBCA-BBTN) to 0.678 (BBNI-BMRI); seeing all five banks might therefore help the model forecast each bank better than seeing it alone.

The sample contains daily adjusted closes for BBCA, BBNI, BBRI, BBTN and BMRI from January 4, 2016 to August 28, 2026. Each stock contributes 2,622 log returns. With a 512-day context, the public amazon/chronos-2 checkpoint produces the 0.05, 0.50 and 0.95 quantiles, using the median as its point forecast. The authors compare four variants. C2-UV reads each stock separately, while C2-MV reads all five together. The remaining variants add day-of-week and month-end flags to C2-MV, or add "market proxies". These proxies come from the same five stocks: a cross-sectional mean return and a 60-session volatility. The authors explicitly distinguish them from LQ45 or index returns.

Monthly forecast origins begin in January 2023. For the ablation, 44 origins and a 5-step horizon produce 1,100 origin-stock-step pairs. The main experiment applies C2-MV at 43 origins to generate 20-step paths. Its random walk benchmark predicts zero, with intervals taken from the context window's empirical 5th and 95th percentiles. Inference uses Diebold-Mariano tests with Newey-West lag 1 and Holm correction, alongside 5,000 origin-clustered bootstrap draws.

What does seeing the other banks buy?

In the ablation, RMSE is 2.1366 for C2-MV and 2.1416 for the random walk. That amounts to five thousandths of a percentage point. C2-UV fares worse than zero at 2.1429. The covariate variants trail both: 2.1443 with calendar inputs and 2.1551 with market proxies. Extra inputs put the model behind the zero forecast.

Even that small multivariate advantage changes with the metric. C2-UV beats C2-MV on pinball loss, 0.4091 against 0.4134, and on 90% coverage, 86.91% against 85.27%. The authors acknowledge that no configuration dominates. Their warning that "correlation alone does not establish that cross-stock information has incremental predictive power" fits the results. Same-day co-movement helps prediction only when a bank's move today leads another bank's move tomorrow. The correlation table establishes the former and tells us nothing about the latter.

The multi-horizon experiment offers little more. At h=1, C2-MV improves on zero by 0.0652 points, with RMSE of 2.6327 against 2.6980. Its advantage shrinks to 0.0077 at h=5, then becomes a loss of 0.0019 at h=20. Aggregate DM statistics are -0.946, -0.624 and 0.550. Every aggregate adjusted p-value reaches 1.0000 after Holm correction.

BBTN supplies the lone stock-level result that looked significant before correction: p 0.0381 at h=1. Correction takes it to 0.5711. At h=20, BBTN's bootstrap CI excludes zero in the opposite direction (+0.0091, CI [0.0014; 0.0171]). Its DM p is nevertheless 0.0513 and its Holm p 0.7176.

The abstract's description, "competitive but limited predictive performance", contains the authors' defence. C2-MV beats Mean-60 at every horizon, including a 0.1267-point gain at h=1. The paper also says this comparison supplies no statistically significant evidence of superiority. Mean-60 itself loses to zero at all three horizons, with 2.7594 against 2.6980 at h=1. Beating it clears a lower bar.

The h=20 figures pool 860 forecasts per stock

Table 6 needs reading before the horizon pattern gets interpreted. A per-stock h=5 coverage figure of 83.26% corresponds to 179/215; an h=20 figure of 85.93% corresponds to 739/860. There are 43 origins per stock. Those denominators mean the calculations pool every step through the stated horizon.

Consequently, h=20 includes 860 forecasts per stock, among them the 43 one-day-ahead forecasts. The h=1 result uses 43 alone. RMSE declines from 2.6327 to 2.0324 as the horizon extends, a pattern consistent with a small and noisy one-day sample. It does not establish that forecasts improve further out. Although the paper explicitly says the ablation pools the five steps, it describes the multi-horizon results as extracted 1-, 5- and 20-session forecasts.

Risk sizing runs into undercoverage

The interval results matter most to a trader sizing risk. Against nominal coverage of 90%, C2-MV delivers aggregate coverage of 76.74% at h=1, 85.40% at h=5 and 87.49% at h=20. For BBCA and BMRI, h=1 coverage falls to 72.09%. High-volatility origins achieve only 73.33% at h=1. The empirical-quantile random walk also falls short at h=1, covering 74.42% to 83.72% by stock. Neither set of intervals holds up in this fat-tailed sample; BBCA's kurtosis is 8.028.

The authors ask readers to consider interval width alongside coverage. They define PIW, a measure of that width, although we did not find PIW values in any result table. Their own qualification is direct: "Wider predictive intervals naturally have a greater probability of containing future observations." Pinball loss accounts partly for sharpness. It also declines with horizon: BBCA's C2-MV loss moves from 0.4394 at h=1 to 0.3268 at h=20. Since this loss pools steps as well, the coverage caveat applies to it too.

Direction stays below 50%

Across the three horizons, C2-MV gets direction right 49.30%, 47.26% and 46.49% of the time. Each falls below a coin flip. The ablation yields 48.18%. With a hit rate of 49.30%, the h=1 RMSE gain of 0.0652 points cannot reflect broadly successful direction calls. At most, a few correctly called large moves account for it.

Mean-60 records 38.60% directional accuracy at h=1. For a trailing mean, a 38.60% hit rate is low enough to raise our suspicion that zero-return days count as misses. We could not find the paper's treatment of those days.

Power is limited with 43 one-day forecasts per stock. The aggregate h=1 gap of 0.0652 points comes with a bootstrap CI of [-0.1852; 0.0410]. That range is too wide for 43 origins to resolve the question. Splitting by volatility leaves 21 and 22 origins, an exercise the authors themselves describe as diagnostic.

We could not run this ourselves.

Our data covers US equities; this experiment requires the return histories of these five specific IDX stocks. Substituting US banks would ask about a different market and a different kind of transferability.

The authors flag further limits: the five-bank sample was selected ex post, and there is no backtest, no costs and no Sharpe ratio. Alongside a best-case RMSE gain of 0.0652-point and one-day coverage of 76.74%, those limits make the paper most useful as a warning. We would change our view if a 0.0652-point RMSE improvement reached significance in a larger universe selected without hindsight. These 43 origins offer no sign of that.