Trend and Hurst gates cut LightGBM crypto coverage from 20.8% to 1.34%

Adding trend and Hurst gates to LightGBM made this crypto signal worse on every forward metric the authors list. Precision, recall, F1, coverage, precision gap and cost-adjusted return all deteriorated. Hassanie and coauthors ran that test themselves. Their revision retains the original title while reporting the gate's failure.

The trade behind the test

The authors build three price ratios from perpetual futures newly listed on Binance in March 2025: MLNUSDT, PLUMEUSDT and SIRENUSDT. Their March to May 2025 files contain 28,157,374 public trade prints. A trade in either leg updates its ratio, carrying forward the other leg's last price. They group those updates into event bars of 20, 50 or 1,000 updates, yielding nine pair/bar configurations.

LightGBM predicts whether the next bar will close above its open. A long requires agreement from three conditions: the model predicts up; a trend cue sees a positive log return and a close above a 10-bar EMA; and the rolling Hurst exponent, a measure of persistence calculated from the last 100 closes, exceeds threshold h. The prespecified threshold is 0.40, below the 0.5 random-walk mark, though the Hurst condition presumably aims to catch persistent stretches. The authors call this combined rule the full AND-gate.

The original assessment used one 60/20/20 chronological split and treated the validation-to-test precision gap as its yardstick. The revision adds an independent audit: three expanding walk-forward folds, four model benchmarks, rule-only controls, ablations, three seeds and 0/10/20 bp cost scenarios. Coverage means the share of test bars receiving a long signal.

A smaller gap on the fixed split

In 7 of 9 cells, the gate narrowed the validation-to-test precision gap. The average reduction was 61.5%. Yet the exact sign and Wilcoxon tests missed significance at 5%, while two 20-event cells worsened. The authors report those qualifications themselves.

They also identify the problem with the yardstick. Precision near 0.5 can remain steady while describing random-like behavior; several test figures are close to 0.5. The apparent standouts,.636 and.600, belong to the 1,000-event bars. Those coarsest bars leave the fewest test observations across roughly two months of history.

Did the gate hold up out of sample?

It collapsed.

Across 27 fold cells, LightGBM alone recorded precision 0.511, recall 0.213, F1 0.251 and coverage 20.8%. The full gate recorded 0.473, 0.014, 0.026 and 1.34%. It surrendered recall without the precision gain a restrictive filter would usually seek.

The original yardstick reversed as well. Mean validation-to-test gap reached 0.024 for LightGBM alone and 0.177 for the gate. The gate's configuration-bootstrap precision interval was [0.400, 0.538], straddling 0.5. With seeds 11, 29 and 47, gate precision came to 0.473, 0.482 and 0.493; LightGBM alone returned 0.511, 0.511 and 0.512.

The trend-only row is awkward for the machine learning models. Its F1 of.386 on 34.1% coverage tops every model; the best model reached 0.251. It fires about a third of the time, with precision of.462. Its gap is.019, and after a 10 bp cost it nets -11.77 bps, the ablation's worst net result. Fewer than half its calls are right.

Validation found no settled threshold. Across 27 cells, it selected 0.40 six times, 0.45 five, 0.50 two, 0.55 five and 0.60 nine. As h rose from 0.40 to 0.60 on the test grid, coverage dropped from 3.16% to 0.22% and recall from 0.032 to 0.002. Precision remained between 0.460 and 0.482.

What a reimplementation would need

The gate itself is closely specified. Its Hurst estimator uses 100 closes, regresses log lagged-difference dispersion on log lag over lags 2 to 20, then clips the result to [0, 1]. Validation selects from thresholds 0.40 to 0.60 in steps of 0.05, using precision first, coverage next and lower h to break remaining ties. Training windows end at 45%, 60% and 75% of each series; each is followed by 10% validation and 15% test. The text gives enough to rebuild those steps.

Some model inputs remain open to interpretation. We did not find the number of return lags, the volatility or RSI windows, or the probability cutoff used to turn a LightGBM score into a positive prediction. The audit describes its hyperparameter budgets only as limited. The authors also say the original code, models and predictions were unavailable, making their own audit a reconstruction. Two implementations of the idea already exist and need not match.

The authors argue for reporting gap, discrimination, coverage and economic value separately. Their argument is sound; their tables still leave questions. The methods promise accepted-signal counts, balanced accuracy and average precision per fold, but the tables we read show coverage without those three measures. At 1.34% coverage, the reader cannot count the trades behind 0.473. The tables give no class balance against which to judge 0.511. We found no short-side test, although the growth-factor equations contain a short leg.

Costs on a synthetic instrument

Before costs, LightGBM alone earned +0.97 bps per accepted signal; the full gate earned -0.16 bps. At 10 bps, their respective results were -9.03 and -10.16. At 20 bps, they were -19.03 and -20.16. The +0.97 bps pre-cost edge for LightGBM alone amounts to about a tenth of one 10 bp round trip. Every model loses after that charge, with XGBoost best at -8.39 bps.

The charge is applied once to the ratio's next-bar open-to-close return. A ratio position entails a long perpetual and a short perpetual, or two round trips per signal, with funding on both legs. The authors explicitly exclude funding, spread variation, impact and synchronized fills. Their as-of price construction brings another distortion: maximum inter-trade gaps reach 257.298 s for PLUMEUSDT and 209.956 s for MLNUSDT. A bar may therefore close using a price from minutes earlier on one leg. Spread, impact and fees for the second leg can only reduce after-cost results already below zero.

Why we could not backtest it

Our platform has no crypto perpetual futures, and one-minute OHLCV is its finest data. Constructing a 20-update ratio bar requires every trade print for both contracts; minute bars cannot supply that signal. The authors provide the data only on request.

The authors conclude: "On the present three-asset, three-month sample, the proposed gate is a proof-of-concept that failed its stronger audit, not a general or deployable trading framework." I agree, retaining their qualification. First archived trades arrive on 21, 22 and 31 March 2025, leaving less than three months of usable history. As the authors note, launch-period trading may have its own regime. A positive result from a nested walk-forward over longer histories, after a two-leg cost model, would make me reconsider Hurst gating. The authors themselves call for that nested design over longer histories.