A trader already running price signals has little reason to add this news portfolio. Its headline Sharpe is 1.12 against 0.66 for a price-only ensemble, with Ledoit-Wolf p=0.463 on the difference. The finding worth keeping lies earlier in the signal: comparing topic attention with its past 63 trading days, rather than 5, improves forecasts for every text reader. The authors acknowledge the portfolio comparison in their abstract. Their case for emphasizing the combined portfolio remains unconvincing for a desk that already trades price.
Reading news without hindsight
Hindsight enters through the text features. An encoder trained on material from within a backtest has seen information from later in the period. A held-out test set for the return model cannot undo that exposure. The authors' proposed remedy is to read the news without hindsight.
Their trade ranks stocks across the top-1,000 cross-section and takes long-short positions. It rests on a relative-return bet: a stock whose returns co-move with a topic gaining news share may behave differently next period from stocks with less exposure. The topic signal follows the narrative asset pricing of Bybee, Kelly and Su. Each article receives weights across topics; their daily averages measure each topic's share of attention. Subtract the trailing mean from today's share, then orthogonalize the resulting attention shock to the market. A stock's narrative exposure is its exponentially weighted covariance with that shock over 252 days (half-life 60). Those exposures go into the return forecast alongside eight price features.
Foley, Hartadi, Prakash, Takkallapalli and Vatsal take the encoder's training date seriously. One released in 2021 has already read years that the backtest will score. They begin instead with NoLBERT, a 109M-parameter DeBERTa-v3 pretrained by Kakhbod and Li (2025) on text from 1976 to 1995 only. Contrastive training on passage pairs from the same pre-2020 article turns it into a sentence encoder, without labels or returns. The run used 230,931 pairs and completed about 0.87 of one pass in 150 minutes on one RTX 3060. Held-out passage matching climbed from 0.08 to 0.89.
Six readers label the same daily sample: two LDA variants, MiniLM, a FinBERT headline classifier, and NoLBERT at 60 and 20 clusters. The sample contains 529,657 articles drawn from 1,255,289 Nasdaq.com articles spanning 2012 to 2024. Eight models forecast relative returns for about 984 of the 1,000 largest US stocks. They rebalance monthly, biweekly or weekly and charge 10bp per dollar traded. Training covers 2013-2019, all choices use 2020-2021, and 2022-2024 remains held out.
The readers finish close together on the question the authors set out to answer. With 63-day shocks, NoLBERT's average monthly test IC is 2.59 against 2.85 for LDA. The biweekly comparison is 1.34 against 2.01; weekly, it is 1.01 against 0.94. No pairwise reader difference in the text-only sets is significant. Topic assignments are steadier for the two NoLBERT readers: 69% and 80% of articles retain their topic across seeds, versus 58% for LDA. The authors call the point-in-time restriction "affordable here" and add that this "is not evidence that it improves forecasts." That reading seems right.
Why does the longer baseline help?
The paper compares 46 to 48 experiment pairs that change only the shock window. Moving from 5 to 63 days lifts test IC by 1.28 monthly, 0.87 biweekly and 0.49 weekly. Annual net long-short return rises by 5.8, 4.7 and 3.9 percentage points, respectively. Sharpe improves in 91%, 80% and 85% of the pairs. Yet the IC gains have t statistics of 1.8, 2.0 and 1.7, below the paper's fixed-b critical values of 2.29, 2.11 and 2.06.
Text alone gives the clearest portfolio comparison. The monthly net decile Sharpe moves from -0.45 with 5-day shocks to 0.86 with 63-day shocks; Ledoit-Wolf p=0.028 for the gap. Adding price features makes the window change worth 0.74 of Sharpe, with p=0.149. The longer window also holds up across the news-source change in 2024. Monthly narrative IC for the 5-day version drops to -2.7 that year. For the 63-day version, it is 1.7 in 2022-2023 and 1.8 in 2024. Differences between the halves are insignificant. Only four windows were tried, the authors note, and the validation advantage was small.
Weekly timing gives me pause. Move LDA shocks back 126 days for a placebo, and weekly test IC is still +0.6, equal to the 63-day narrative exposures. At the monthly horizon the placebo is -0.0 against +1.8. That pattern suggests narrative timing carries more information monthly than weekly. The comparison also uses different sets of experiments: LDA alone for the placebo (8 experiments), all six readers for the narrative average (48). At a weekly horizon, these exposures look much like generic return covariance.
The price-only rival remains
The monthly equal-weight combination uses six readers and eight models on price plus text, with 63-day shocks. It reaches IC 3.9 (t=2.91), then posts net Sharpe of 1.12 in the decile book and 1.19 in the buffered book. The buffered version earns 12.4% a year net at 10.4% volatility, with an 8% maximum drawdown and 60% monthly turnover. The authors describe its result against zero as significant before adjusting for the search (decile p=0.045). That comparison holds. A desk with existing price signals needs the comparison with price instead.
Price alone delivers IC 3.3, decile Sharpe 0.66 and buffered Sharpe 0.53. Against that ensemble, the combination's Sharpe gap has Ledoit-Wolf p=0.463 for deciles and 0.267 for buffered positions. Its deflated Sharpe ratios are 0.04 and 0.06. The authors disclose that they formed the combinations after the main experiments, which makes even p=0.045 an overstatement of the evidence. The test period has limited power: with a Sharpe standard error around 0.58, reaching 80% power requires a true Sharpe near 1.7.
Capacity makes the result harder to use. Market-value weighting cuts the monthly combination's Sharpe from 1.12 to 0.03. The only reported alpha I found has a three-factor t of 2.21, while momentum and reversal are already among the features. Costs bite at higher frequency as well: weekly, the same combination falls from gross Sharpe 1.02 to 0.30 net.
What validation selected
At every rebalancing frequency, the experiment with the best validation IC uses an explainable boosting machine (EBM). Averaged across feature sets, EBM also leads validation IC among models at 2.4, 2.0 and 1.7. Its test IC is close to zero. The validation pick records net test decile Sharpes of -0.23, -0.27 and -1.08.
The authors inspect the price-and-LDA-text EBM, the overall pick at the monthly and weekly frequencies. Its ten pairwise terms account for 98%, 58% and 79% of the model. Most involve a firm's news share in a topic. For 40% to 50% of firms, that feature is imputed from a random stock. Removing the pairs without refitting changes Sharpe from -0.23 to 0.00 monthly and from -1.08 to -0.41 weekly.
Even the seed matters as much as sampling error over three years of returns. Across ten seeds, monthly XGBoost on price and LDA text ranges from -1.00 to 1.47. An earlier return extract omitted returns for stocks nearing their exit from the universe. On price alone, that omission generated a spurious weekly net Sharpe of 1.56. The authors identified and fixed it. Their diagnostic work is the paper's most transferable section.
Our weekly LDA run
We could not rebuild the paper's experiment. Our coverage begins around 2020, leaving its 2012-2019 topic fitting and return training beyond what we can reproduce. NoLBERT is unavailable in our tooling, and our news feed differs from FinMultiTime/FNSPID. We ran a forward-shifted, point-in-time LDA attention strategy. This run says nothing about NoLBERT relative to other readers.
We used:
- A 60-topic LDA frozen after 252 sessions of roughly 2020 news, with a one-session availability lag.
- 63-day shocks residualized on SPY and 252-session exposures (half-life 60).
- An expanding ridge forecast.
- An equal-weighted weekly decile long-short on a year-matched top-1,000 universe, restricted to stocks with a firm article in the prior 126 sessions.
We charged commissions of $0.004 a share on every fill and used closing-auction fills. Short borrow and market impact were unmodeled. From 2022-01-07 to 2024-07-01, the run returned -19.91% in total, with a 34.50% maximum drawdown and net Sharpe -0.73.
The paper's 1.12 headline comes from a monthly, 48-model combination on different data and with a different method. The gap therefore cannot serve as a failed replication. Closer paper comparisons are the weekly price-plus-text combination at 0.30 net Sharpe and the median weekly decile Sharpe across base experiments at -0.34. Weekly ridge is also -0.34. Our -0.73 is lower still, for reasons we cannot fully explain.
Three confirmed differences each reduce results in the paper's own figures: our trading is weekly, we use one model, and our price features leave out 12-1 momentum and 5-day reversal. Two further differences likely matter. Our coverage gate excludes names the paper retains by imputation, while our LDA and ridge were trained on about a year of COVID-era news and labels. Ending in mid-2024 may also have hurt, given the paper's stronger price signals in 2024. The placebo result makes weekly trading a particularly unpromising place to look for narrative timing. Our -0.73 speaks first to this implementation, a single automated pass rather than a verdict on the authors' work.
A longer test would change my view if the 63-day combination beat the price ensemble and its legs survived cap-weighting. Over 36 months, the paper shows the longer window lifting IC for every reader without clearing its own fixed-b threshold. It leaves the case for news paying beyond price unproved.
Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.