AQAI QuantAI research lab for systematic strategies

Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysisOur backtest

AlphaRLM's S&P 500 gain disappears with Ridge

An LLM writes price-volume factors from finance abstracts; the US result depends on 1,000 factors and depth-8 trees.

2026-10-14 · 7 min read · Symbolic factor discovery · US large-cap equities

Reviewing: AlphaRLM: Aligning Language Models for Literature-Grounded Symbolic Factor Discovery · Jie Huang, Yuecheng Hong, Yao Chen et al. · Read it on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

Literature-Grounded Symbolic Factors with Training-Signed Endpoint Selection

Backtest period 2020-01-01 to 2024-07-01 · hypothetical, net of modelled costs

Why these figures are not the paper's (3)

Run on a different market than the paper

Evaluate only the paper's US-equity setting, using a year-specific market-cap-screened universe rather than exact historical S&P 500 membership. Cross-sectional price-and-volume factor selection still applies, but this does not replicate S&P 500 membership or the Chinese-market results.

The paper's own figures describe its universe and do not carry over to ours.

This is not a replication of the paper

  • The curated historical literature corpus and 8B/14B DAPO policy-training capability are not among the stated resources. Use classical symbolic-expression search with the paper's executable-validity and grouped-return-separation objectives; this tests a substitute, not AlphaRLM's literature-conditioned model or its claimed advantage.

The figures below measure what we could run, not the paper's own method, so they are not evidence for or against its claim.

Our own audit found this run does not follow the paper faithfully (13)

  • Eq 4: Δ̂_K(f) = max_g μ̂_g(f) − min_g μ̂_g(f).: Selection uses signed endpoint S(f)=μ̂_K(f)−μ̂_1(f), not an all-group maximum-minus-minimum. (invalidates: Paper Table 1 AlphaRLM S&P 500 AR and MDD; Paper Table 2 TMB-setting AR and MDD)
  • Eq 5: r_tmb(f) = 100 Δ̂_K(f) if valid, and 0 otherwise.: No TMB reward or TMB policy training is run; signed pooled endpoint separation selects a factor. (invalidates: Paper Table 1 AlphaRLM S&P 500 AR and MDD; Paper Table 2 TMB-setting AR and MDD)
  • Eq 6: r(f) = α r_valid(f) + β r_tmb(f).: Validity is a rejection gate and signed endpoint separation is the selection statistic; there is no combined RL task reward. (invalidates: Paper Table 1 AlphaRLM S&P 500 AR and MDD; Paper Table 2 TMB-setting AR and MDD)
  • Eq 7: R_i = r(f_i) + r_length(o_i).: No rollout reward or length penalty is calculated. (invalidates: Paper Table 1 AlphaRLM S&P 500 AR and MDD)

9 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total 105.8%Jul 2024
Sharpe
0.83
Total Return
105.8%
Max Drawdown
-50.8%
CAGR
17.5%
Volatility
28.3%
Beta vs SPY
1.09
Trades
43,938

What the paper reports for its own strategy

  • AlphaRLM-8B CSI 300: annualized excess return 12.14% (0.1214±0.0011), MDD -6.15%, IC 0.0256, RankIC 0.0424, ICIR 0.1586; test 2021-2024; net of transaction costs (0.05% open/0.15% close); 5 seeds
  • AlphaRLM-8B CSI 1000 (CSI 300-trained factor pool): annualized excess return 17.45%, MDD -11.98%, IC 0.0302, RankIC 0.0414, ICIR 0.2398; test 2021-2024; net of costs
  • AlphaRLM-8B S&P 500: annualized excess return 6.94%, MDD -14.84%, IC 0.0021, RankIC 0.0031, ICIR 0.0158; test 2021-2024; net of costs (cost schedule stated in CNY terms)
  • AlphaRLM-14B CSI 300: annualized excess return 13.38%, MDD -6.05%; test 2021-2024; net of costs
  • AlphaRLM-14B CSI 1000: annualized excess return 17.46%, MDD -14.35%; test 2021-2024; net of costs
  • AlphaRLM-14B S&P 500: annualized excess return 8.08% (±1.25%), MDD -14.77%; test 2021-2024; net of costs

AlphaRLM's 8B S&P 500 book earns 6.94% annualized excess return with depth-8 XGBoost trees. Keep its selected factors and switch to Ridge regression, the other predictor the paper tests, and it trails the index by 3.90% a year.

That difference governs my reading of Huang, Hong and their coauthors. The US result comes from a selected factor pool, a nonlinear predictor and a daily trading rule. The authors present it as evidence for the factors themselves. Their diagnostics make a stronger case for that interpretation in the Chinese markets than in US large caps.

From abstracts to trades

Qwen3 reads the title and abstract of a finance paper, then writes one executable Qlib expression. The model comes in 8B and 14B parameter versions. It works with six daily fields ($open, $close, $high, $low, $volume, $vwap) and a fixed operator list, with no seed factor to mutate.

A valid expression receives a reward of 1, plus 4 × 100 × TMB. Validity means it parses and its historical values stay fixed after the addition of 1 day, 1 week, 1 month or 1 year of future data. That future-extension test is a sensible, cheap check for look-ahead. TMB measures separation between groups of returns. Each day, the authors sort stocks into 100 factor quantiles, pool next-day returns within each quantile across 2018 and 2019, and subtract the lowest group mean from the highest, regardless of where those groups fall in the ranking. DAPO is the group-relative policy-gradient method used to train the model: it samples 8 completions per prompt and favours those that beat their siblings.

The literature supplies the starting idea. An abstract on price impact, asymmetric volatility or higher moments gives the model an economic prior; market feedback rewards the price-volume expression that works. The ablation supports a role for that prior. When DAPO trains on non-financial text instead of finance abstracts, excess return falls from 12.14% to 2.13% on CSI 300 and from 6.94% to 2.90% on the S&P 500. The corpus contains 4,443 papers published from 2015 to 2019: 1,720 training prompts, 430 validation prompts and 2,293 held-out generation prompts.

After generation, the authors screen the pool for validity and rank it by XGBoost feature importance on 2015 to 2019. They retain 1,000 factors. A second XGBoost, with 600 trees, depth 8 and eta 0.04, predicts rank-normalized next-day returns. The daily book holds the top 50 names while excluding the bottom 5 from its candidate pool. Results cover 2021 to 2024 after 0.05% opening and 0.15% closing costs. Across all three markets, the 14B and 8B variants place first and second for annualized excess return. On the S&P 500, they post 8.08% (±1.25%) and 6.94%, respectively; the strongest baseline, FactorMiner, posts 3.16%. Their drawdowns are -14.77% and -14.84%. The paper annualizes the arithmetic mean of daily excess returns by multiplying it by 252, and measures drawdown on summed rather than compounded excess returns.

What survives a linear predictor?

The S&P 500 correlations barely register. The 8B pool posts an IC of 0.0021, a RankIC of 0.0031 and an ICIR of 0.0158 alongside its 6.94% excess return. The authors acknowledge the mismatch: return advantages do not come with uniformly better IC or RankIC, they write, and marginal predictive correlations alone do not fully account for portfolio performance. They use the Ridge run to argue that "nonlinear interactions learned by XGBoost are not the sole explanation" for the gains.

For the A-share universes, Ridge preserves the AR ranking. The 14B pool remains first, with 8.32% on CSI 300 and 9.58% on CSI 1000. The S&P 500 table tells a harsher story. AlphaRLM-8B leads at -3.90%, ahead of the 14B pool, while every method in the table loses. The authors report the negative figures and claim a relative advantage: this run asks whether their lead depends entirely on XGBoost-specific nonlinear interactions. The lead survives. The positive 6.94% does not survive the predictor change, so the Ridge result offers no positive linear signal from the US factors. First place here means the smallest loss.

On US large caps, the gain therefore appears to depend on interactions among 1,000 weak features in a tree model fitted on 2015 to 2019. The authors also point to a factor-level result: AlphaRLM-8B has the highest median single-factor TMB on the S&P 500 test period, 2021 to 2024. TMB is the training objective, though, and its max-minus-min construction catches sample extremes. A high median offers limited evidence for a tradable linear signal. We did not find style or beta neutralization in the paper, leaving open whether size or short-term reversal drives the result.

The selection inside the result

The language model's output passes through two selections based on noisy in-sample estimates. First, TMB chooses the maximum and minimum among 100 group means. CSI 300 is the A-share universe used to train the policy; with 100 groups, each has roughly 3 stocks per day. The reward pools group observations over 2018 and 2019, so a mean uses two years of stock-days rather than just 3 names. Even with that pooling, selecting the extremes of 100 estimates creates upward bias. The authors derive a finite-sample bound whose estimation error increases with the number of groups. They give the countervailing cost too: coarser grouping "can average away localized non-monotonic structure". For them, the group count trades resolution against error. They call the bound "an interpretive reference rather than a distribution-free guarantee" and TMB "a task-level surrogate" for factor usefulness, short of a full characterization.

The ablations nevertheless favour 100 groups. With 30, excess return falls to 8.53% on CSI 300 and 2.11% on the S&P 500. An endpoint reward, the absolute top-quantile minus bottom-quantile spread, produces 9.68% and 0.77%. Its CSI 300 IC is higher, 0.031 against 0.026, while its drawdown reaches -15.60% against -6.15%.

Next comes the factor count, which the authors say they fixed at 1,000 using training-period information. In the S&P 500 test, 1,500 factors yield 2.27%; 2,000 yield -0.33% and a -32.47% drawdown. CSI 300 peaks at 360 factors, with 12.65% versus 12.14% at 1,000. Extending the same ranked pool takes the US figure from 6.94% below zero.

The training reward settles near 21 in both markets. The policy has learned its objective; how much of that objective reflects noise remains open.

Our run on 300 US large caps

We could not run AlphaRLM itself. We lack the curated 2015 to 2019 literature corpus and the capacity to train 8B or 14B policies with DAPO. We instead searched a deterministic library for classical symbolic expressions, retaining the paper's future-extension validity check. Our reward used an endpoint version of grouped-return separation: 30 groups and a signed top factor-quantile mean minus bottom factor-quantile mean on 2015 to 2019 open-to-open returns. We then froze the sign. This is closer to the paper's endpoint ablation than to TMB. We traded one expression rather than feeding 1,000 to XGBoost, and changed the universe to the 300 largest non-ADR US stocks from an annual capitalization screen instead of historical S&P 500 membership. This adaptation says nothing about AlphaRLM's claimed advantage.

From January 2020 to July 2024, our book compounded at 17.46% a year. Its volatility was 28.33%, its Sharpe ratio 0.83, and its beta to SPY 1.09. Maximum drawdown reached -50.75% across 43,938 trades. The paper gives no Sharpe ratio for comparison. For AlphaRLM-8B over 2021 to 2024, it reports 6.94% excess return and a -14.84% drawdown. Our return is higher, and our drawdown is far deeper. Our figure is an absolute return; theirs is index-relative, from a different method and universe.

Market exposure explains part of the gap. At 1.09 beta, much of our 17.46% comes from the market, which the paper removes from its excess-return measure; that exposure also deepens our drawdown in selloffs. Our window includes the 2020 crash, used only for validation in the paper. Trading one expression concentrates the book more than a 1,000-factor tree model, consistent with our 28.33% volatility. We see two further likely contributors. The capitalization screen has not been verified as point-in-time and could contain survivorship. Our charge of $0.004 a share is probably cheaper for large caps than 20 basis points round trip, flattering our return. Position size may matter as well: our specification permitted up to 4.0x leverage, and any use of it could have increased both return and drawdown. Specified next-open fills also conflict with the recorded market-on-close execution. We cannot calculate our index-relative return and therefore cannot fully account for the gap.

Where implementation remains unclear

The paper specifies the model reward, all four validity horizons, DAPO clip ratios of 0.2 and 0.28, and the complete XGBoost settings. The US trading assumptions are less settled. The only cost schedule we found includes a 5 CNY minimum fee and 100 million CNY in starting capital; we did not find a separate US schedule. Nor did we find whether the WRDS constituent lists are point-in-time. The five seeds behind the ±0.0065 band for the 8B S&P 500 figure appear to change the downstream model. We found no reported variation across RL runs or generation draws. The 2019 cutoff applies to the prompts, while Qwen3's pretraining may extend beyond the 2021 to 2024 test period; we found no discussion of that possibility.

I would rebuild the US downstream stage with Ridge and style neutralization before treating the literature-conditioned RL as verified. Until a US run produces positive linear IC from these factors, I would size the 6.94% as a claim about a language-model-fed tree with 1,000 features.

Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

For each year, screen the 300 largest eligible non-ADR stocks by that year's capitalization.
Using 2015–2019 data only, validate candidate expressions and their historical-score invariance to future data extensions.
On each training date, sort finite scores into 30 groups; label each stock with its next-open-to-following-open return.
Pool stock-day labels within each endpoint group. Select the valid expression with the largest absolute signed spread (highest-group mean minus lowest-group mean); freeze its sign and expression.
After each test-session close, score and orient eligible stocks. Exclude the bottom five, target the top 50 at 2% each, and exit names that leave the top 50 or annual universe.
The specified strategy fills at the next actual adjusted open and skips orders without that price; the platform cost record instead identifies MOC execution, an unresolved discrepancy.