Song and Xiang's headline holds because of the qualifier "that used news". An eight-week filter puts their direct LLM portfolio policy ahead of its own market-only run at every delay: 20.16%, 18.35% and 18.43%, versus 18.02%. The advantage at four weeks is 0.41 points. There, the filtered LLM also outperforms every RL policy that used news (17.81%, 18.02% and 18.03%). Market-only RL earned 19.07%.
The paper gives no interval for filtered versus market-only. All its intervals against News cross zero.
Ten Dow names, shared signals
A favorable 10-Q can be worth trading before prices finish adjusting. After a month, its information may already be in the price. The paper cites Tetlock's work on investors' treatment of stale information and asks a practical question: should a portfolio model receive an aging signal with its age attached, or should the signal disappear after a cutoff?
GPT-4.1 mini reads each periodic SEC excerpt once, assigning +1, 0 or -1 and supplying a verbatim supporting quote. There are 277 excerpts, each 800 to 1,000 words, covering 2019 to 2025. Of those, 119 belong to the test cohorts. The stocks are AAPL, AXP, BA, CAT, CSCO, CVX, HD, JNJ, KO and VZ.
Each week, the policy chooses among 57 long-only actions: equal weight, ten single-name 40% tilts, 45 two-name 30/30 tilts, or HOLD. It decides using the week's last close and trades at the next week's first open. Buying or selling costs 10 bp per dollar. The stored signals feed two kinds of decision maker. A Double DQN is retrained with ten seeds per RL setting in each of three expanding folds, producing 120 trained policies across the four settings. GPT-4.1 mini instead selects an action directly from anonymized numeric rows.
The market-only baseline has three news treatments alongside it. News supplies just the signal. News + Age supplies the signal and two timing fields: weeks since publication and weeks since the policy received the report. Filtered News sets a signal to zero when its report exceeds eight weeks in age, without supplying timing fields. Reports reach the policy after an additional zero, one or four weeks. Testing covers 2023, 2024 and 2025.
Across those three delays, the filtered LLM returned 20.16%, 18.35% and 18.43% annualized net. Unfiltered news gave the same model 17.60%, 16.19% and 16.02%.
What does 18.43% beat?
The complete four-week LLM row changes the emphasis. Returns were 18.02% with market data alone, 16.02% with news, 17.17% with news and age, and 18.43% after filtering. News therefore cost two points; the filter recovered those points and added 0.41. At this delay, the authors report that filtering removes roughly 70% of nonzero signals. Their best LLM news policy receives about 30% of the nonzero signals available to News.
The RL benchmark deserves equal care. The paper says the filtered LLM outperformed "all RL strategies that used news". Those final words set the boundary of the claim. Market-only RL returned 19.07%, which the authors explicitly acknowledge exceeds 18.43%. They also acknowledge that adding news does not always improve on market data alone. Even so, the abstract and conclusion lead with the comparison confined to news-using RL.
Passive equal weight earned 17.63%, with a 1.13 Sharpe and 1.09x annual turnover. The filtered LLM delivered a 1.12 Sharpe at 19.03x turnover. Its drawdown was 10.83%, against 10.44% for equal weight. Sharpe ratios appear only for the four-week delay, where every active policy falls short of equal weight's 1.13. The filtered LLM comes closest at 1.12.
Annual results weaken the case for the 2.41-point filtering gain over News. For the LLM, Filtered minus News is +4.34 in 2023, -3.93 in 2024 and +6.83 in 2025. The corresponding 95% intervals are [-4.90, 18.77], [-24.10, 13.41] and [-6.60, 20.02]. The filter lost in one year out of three. Across all 36 primary annual timing and filter comparisons, every eight-week-block interval includes zero.
RL changed its trades. Returns barely moved.
I would give this part of the paper to anyone connecting LLM signals to a book. Age the same reports by twelve weeks, leave everything else unchanged, and RL with News + Age chooses a different portfolio in 52% of cases. That figure averages the ten trained policies over 39 validation observations in the 2024 fold.
At the four-week delay, the return gain is +0.21%, with a 95% interval of [-3.73, 4.29]. Filtering adds +0.22%, with an interval of [-2.93, 3.44]. Timing inputs reduce returns at zero and one week: 15.41% versus 16.22%, and 15.11% versus 16.95%.
The authors deserve credit for stating the limitation themselves. Changed decisions demonstrate "sensitivity to timing information, without establishing that RL systematically reduces the influence of older signals." Their appendix makes the instability clearer. Change rates for individual policies range from 25.6% to 74.4%. In every year, the four-week timing contrast changes sign across seeds; some trained policies assign more signal-aligned weight as reports get older. When a network's response to a feature reverses with its seed, it has learned nothing stable about that feature. The synthetic market tells the same story: timing contributed +0.35% [-0.09, 0.79] with fast-changing hidden states and -0.32% [-0.71, 0.08] when states persisted.
Cheap enough to keep, too early to trust further
I would adopt the eight-week filter for a narrow reason: it takes one line of code. At four weeks, its pooled effect was positive for both decision makers, +0.22 points for RL and +2.41 for the LLM. But the LLM filter lost 3.93 points in 2024, and the RL filter lost 1.41 in 2023. The paper gives too little support for deploying anything beyond that.
The evaluation is retrospective, as the authors disclose. They had used 2023 for validation in an earlier analysis and had already examined 2024-2025. At the four-week delay, most active policies generated annual turnover of 19x to 25x. Costs reached 2.73% for RL Filtered News. LLM News + Age had the lowest turnover, 13.01x. In the fee replay, RL News + Age falls from 18.02% to 12.32% at 30 bp. The replay covered all ten methods, although RL News + Age is the only active policy for which the paper reports a 30 bp return.
Extraction quality on actual filings remains unresolved. Only 228 of 277 extractions (82.3%) passed the format and evidence checks. The 49 failures were converted to zeros, making them indistinguishable from neutral signals. Without human labels, the authors concede, semantic accuracy cannot be assessed. There is also the extractor's date: a 2025-04-14 snapshot reads filings from 2023 and 2024 that may reveal their issuers. The prompt prohibits outside knowledge; anonymization occurs only when the portfolio decision is made.
We could not rerun this. The method requires historical narrative text from 10-K and 10-Q filings. Our data contains structured statement values, news, press releases and transcripts. Substituting news would alter the source and the arrival cadence. An eight-week cutoff would then filter a different signal.
I would change my view if a filtered LLM beat its own 18.02% market-only run with an interval clear of zero, across more than ten stocks and three years.