Bitcoin forest with AUC 0.5106 backtests $8,491

At its reported operating point, this Bitcoin forest catches 99.50% of true uptrend hours, with 41.38% precision. That looks close to a standing long. Fadilah, Yustiana and Nugraha acknowledge the weak discrimination in the abstract: an AUC-ROC of 0.5106 means the model "can hardly distinguish between the two classes individually." Their trading result rests on what comes next. With a probability threshold, a 1:3 risk/reward bracket and a trailing stop, the backtest turns 10,000 USD into a net profit of 8,491 USD. They chose the threshold on the test window.

What goes into the forest

The authors use hourly BTC/USD bars from MetaTrader 5, spanning 1 January 2017 to 2 March 2026. The sample starts with 80,228 bars and falls to 79,980 when lag and rolling windows shed their warm-up rows. Triple-barrier labelling assigns each bar a binary outcome: 1 for uptrend, 0 for downtrend or sideways. We did not find the barrier widths or holding horizon in the paper. Uptrend accounts for 41.9% of samples (33,613), making it the minority class.

From OHLCV alone, the authors make 106 features in nine groups. The biggest groups contain 24 lags, 18 trend indicators and 18 rolling statistics. Their scikit-learn random forest has 200 trees, depth 30, considers 40% of features at each split and weights the uptrend class 2.0. Tuning uses a 3-fold time-series split. The model trains on 2017 to 2023, uses 2024 for validation and tests on 1 January 2025 to 1 March 2026.

The importances suggest a forest attentive to recent chop. Its leading feature, rsi_14_std_168 at 3.77%, measures the one-week (168-hour) dispersion of a 14-period RSI. Third-ranked atr_pct_14_mean_168, at 2.92%, measures the week's average range. Rolling statistics occupy 7 of the top 15 places; volume features are near zero. Raw price levels also rank high: sma_200 (3.03%), sma_100 (2.81%) and sma_50 (2.56%). Because those levels are non-stationary, a tree can locate the era rather than identify a setup.

Was the test set still a test set?

The threshold scan runs from 0.01 to 0.80 in 0.01 steps "pada data uji", on the test data, despite the reserved 2024 validation year. The authors select 0.35 to 0.37 as the practical range, where profit factor runs from 1.30 to 1.39. They also construct five risk profiles with cutoffs from 0.01 to 0.35. The scan finds a profit factor of 6.00 at 0.55, a figure the authors call theoretical because zero trades produced it. Selecting 0.37 on the test window makes the headline 8,491 USD in-sample with respect to the threshold.

At 0.01, the model makes 10,173 trades. At the F1-optimal 0.08, it makes 10,172. The test spans about 14 months, roughly ten thousand hours, so the 0.08 setting is long nearly every hour. Its reported recall is 99.50%, with F1 at 0.5856. Given the full-sample uptrend share of 41.9%, an always-uptrend classifier would have an F1 near 0.59. The five cross-validation folds fall between 0.5821 and 0.5853, around that same trivial level; their closeness says little about generalisation. The stated standard deviation is 0.0023, about double the roughly 0.0012 to 0.0013 implied by the five listed values.

Accuracy raises another problem. Recall of 99.50% paired with precision of 41.38% means nearly every bar gets an uptrend call, putting accuracy near the up-share. The XGBoost comparison table gives the forest 41.20%, exactly in that vicinity. The headline figure is 58.65%, close to the full-sample down/sideways share of 58.1%, which an always-down call would achieve on that sample.

The two accuracy figures cannot both describe one model.

The authors say every pre-set target was met. Their AUC target was 0.51.

A different threshold in the trailing-stop row

The trailing stop starts at +1.5% and follows the high by 1.0%. The authors credit it with raising win rate from 33.78% to 45.08%. Yet the compared rows change the entry threshold as well: 0.35 without the trail, 0.37 with it. A higher cutoff should allow no more entries, while the reported count climbs from 148 to 264. Faster exits could free a slot sooner under a one-position-at-a-time rule. We did not find the position rule described.

The trail's other figures are harder to celebrate. Net profit drops by 487 USD, from 8,978 to 8,491. Profit factor slips from 1.39 to 1.30 and Sharpe from 1.14 to 0.95; max drawdown grows from 16.07% to 24.78%. The authors take the added trades and higher win rate as consistency. Their table shows worse risk-adjusted return on every measure.

A 1:3 bracket needs a 25% hit rate to break even. With losers costing a full 1R, the 45.08% win rate and 1.30 profit factor imply an average winner about 1.6 times the average loser. The target is 3R, so the average winner gets about halfway there. That 1.6x average winner suggests many of the 96 trailing exits (36.4%) finished well short of 3R. We did not find per-exit results in the paper.

We did not find transaction costs, spreads or funding in the paper. Across 148 to 264 trades, costs would reduce the result. Across 10,172 trades yielding 2,462 USD, they could consume it. We also did not find a buy-and-hold comparison for January 2025 to March 2026. Buy-and-hold is the natural benchmark, especially at 0.08, when the model trades 10,172 times over roughly ten thousand hours.

The XGBoost comparison separates two near-chance models by 69 USD: net profit is 2,462 against 2,393, AUC is 0.5120 against 0.5110, and each has a profit factor of 1.41.

The threshold should have been set using the 2024 validation year, then frozen before 2025. At that cutoff, the forest needed to beat buy-and-hold and an always-long entry with the same 1:3 bracket, after costs. Because the authors chose the threshold on 2025-26 data, their 8,491 USD has no out-of-sample counterpart.