Bai and Bian give LLM factor miners machinery worth copying, without showing that it will make them money. Their warning is plain: "QBX does not manufacture alpha." The gap matters to anyone looking for a tradeable signal. The headline RankICIR frontier of 0.35 to 0.44 comes from selection on the test window. Typed-only books selected on validation deliver 0.236 to 0.283.
QBX compiles alpha programs. Each strategy is a directed graph assembled from a fixed registry of 95 typed components, including indicators, transforms, risk rules and seven execution models. Output ports connect to input ports. Whether an LLM, genetic search or human proposes the graph, it either fails with a diagnostic or compiles into an intermediate representation. This records the types and execution schedule, along with a signed availability offset for each node. A node also receives a SHA-256 address derived from its block, parameters and parents' addresses. Those addresses identify cached results: change one window in a child strategy and only the downstream work needs repeating. The root address supports deduplication and a trial ledger. That ledger supplies the trial count a deflated Sharpe needs and seldom gets.
The cost argument is persuasive. Compilation and deduplication take 56.7 µs per candidate; the median LLM call takes 1.45 s. Each call can therefore seed roughly 400 legal candidates through local search. The authors use CSI500 daily OHLCV from BaoStock and score a 20-day forward-return target with daily rank IC and its ICIR. Training finishes on 2020-12-31, validation covers 2021-01 to 2022-06, and testing begins in 2022-07. The authors explicitly limit their results to validity, reuse and search efficiency within the registered vocabulary, "not an increase in the information available from the market panel."
We could not rerun any of this. Our platform lacks a historical Chinese A-share universe, and moving the compiler to US equities would test a different market.
What does validity buy?
The headline ladder compares six arms over 10 seeds and 300 iterations apiece. Evaluation-valid proposals rise from 26.8% for mutated strings through 36.4% for the grammar-only control to 97.7% for typed programs. That last figure is nearly guaranteed by the construction: search returns only children that compile. As the authors put it, high typed validity "follows from compiler closure."
The decomposition tells us more. Grammar legality explains 13.5% of the gap (+9.6pp, CI [8.4, 10.8]); type and availability checks explain 86.5% (+61.3pp, CI [60.0, 62.6]). Distinct valid factors per run increase from 7.1 to 38.0 to 204.9. Quality improves less. Typed search beats grammar by +0.132 in validation ICIR, with a 95% interval of [0.02, 0.25]. This is a validation-window gain whose lower bound is 0.02.
A broken formula wastes a backtest slot. Validity tells us nothing about turnover, and we did not find a turnover figure anywhere in the paper.
Tomorrow's close gets caught at compilation
The compiler catches the leak it was built to detect. Each value carries an offset φ, recording when it becomes known relative to its index. Substituting a one-period future shift for a causal return input raises φ to +1. Compilation then fails before the panel loads. Conversely, a value indexed T+7 and published at T has φ = -7, making alignment by seven periods legal.
This produces the paper's strongest single result. The static checker rejects 6/6 planted look-ahead factors; a train/test split catches them 0/6 times. The authors explain why: look-ahead can inflate both train and test scores, so the split cannot detect it.
Raw LLM output in the wild leaked at 0.17% (2 of 1,151). Among admitted candidates, 0 of 23,342 violated a registered contract. The authors draw the boundary themselves: "Zero admitted contract violations is therefore not a claim of zero backtest leakage." The trusted base still includes timestamps, revisions and universe membership.
Their extended data end in 2026-05 and, as they disclose, use current-membership snapshots. The six-arm ladder uses that extended panel. Validity counts survive this choice unchanged, but the ICIR deltas come from a universe that excludes names which left the index. The availability check does not address that problem. A warning-only prototype flags ranks lacking point-in-time universe provenance, with testing confined to planted cases.
A 7.5x replay gain, confined to one AAPL series
Incremental replay of a search lineage ran 7.50x faster than full recomputation, producing 0 mismatches across 1,760 comparisons. The data comprise 1,738 one-hour AAPL bars from 2025. One stock. Synthetic graphs with 246-node structures required 21.4x fewer node evaluations for trees and 48.4x fewer for wide DAGs.
The CSI500 ladder gives a less flattering wall-clock result. Memoization saved 36.6% of node evaluations while adding 16.9 s of elapsed time. The authors blame cheap kernels at this scale and argue that avoided evaluations are the appropriate measure. That defence is fair for a 300-iteration run. The 7.50x speed headline nevertheless rests on the single AAPL series.
For its identity comparison, the paper uses a bolt-on AST cache. It remains slower than running without a cache until about 1,100 candidates, whereas QBX reaches 1.69x in the same test by assigning identity at construction.
The authors acknowledge the absent control. They call a let-bound, hash-consed AST "the correct strong baseline," and say that until it is run, the 7.5x cannot be apportioned among content identity, ordinary memoization and the evaluator implementation. They deserve credit for making that limitation explicit.
Typed-only books lose breadth and performance
The typed arm reaches a best test RankICIR of 0.35 to 0.44 from two LLM calls, roughly twice the string-mutation result. Every point on this frontier is selected on the test window. The paper calls the curve a diagnostic. Against strings, the typed edge is +0.10 and +0.12 at K=10 and 20 calls, disappearing at K=40. The mean edge on CSI300 is +0.061. A string kernel with 13-action coverage closes the gap at K=10 over three exploratory seeds. At K=20, typed search remains ahead without seed-significance.
Validation selection changes the result. Typed-only books produce test RankICIR of 0.251, 0.283 and 0.236 at K=10, 20 and 40. LLM-only books achieve 0.316, 0.321 and 0.343. Effective breadth spans 1.00 to 1.93 for typed search, against 4.65 to 9.38 for LLM proposals. After sign alignment, two typed composites agree to four decimals.
The authors report the shortfall in their own contributions list and propose the hybrid in response. They also caution that action-set coverage may explain part of the typed-book mode collapse, alongside proposer behavior.
The hybrid repairs the shortfall. Combining LLM proposals for breadth with typed local search for depth beats LLM-only by +0.036 ± 0.011 at K=20 and +0.038 ± 0.010 at K=40, across 3 of 3 seeds. At K=10 the gain is +0.006, inside the noise. The hybrid is therefore the version worth attention. Its support amounts to three exploratory seeds of pre-cost ICIR.
The target adds another qualification. Daily scoring of a 20-day forward return means consecutive targets share 19 days. In a preliminary calibration, the authors find that the standard deflated-Sharpe construction is anti-conservative at overlapping horizons. They declare finance realism, including costs and capacity, outside scope. We did not find a turnover figure for any book.
A hybrid book's long-short return net of costs over the 2022-07 test window, with more than three seeds, would move me.
Keep the ledger
Exact-string deduplication eliminates 40.1% of raw LLM output, then 0% after admission. Bounded saturation, which removes equivalent forms before evaluation, eliminates 13.8% of post-admission arrivals. This replaces exact-string dedup's 0%. Among canonically distinct survivors covered by the check, 27.9% turn out value-identical after evaluation. The paper uses that finding to correct breadth counts; it saves no evaluations.
The Hubble adapter turns 1,000 emissions into 250 canonical identities. Across 12 of 12 corpora, the ledger reproduced independently recounted trial denominators. For a desk using LLMs to mine factors, an honest trial count and a static look-ahead check are worth more than the frontier chart. Both stand independently of that test-selected frontier.