A headline signal with out-of-sample AUC 0.50 is a thin reason to trade this policy. The critic supplies the only hint of statistical support, and even that is uncorrected. In the two-stock group, its estimated main effect, averaged over both no-critic comparisons, is +0.0125 of Sharpe, with an uncorrected interval of [+0.0008, +0.0250]. The news-reading component gives the paper its title, yet no effect from it survives the authors' own tests. Pan and co-authors acknowledge this.
The policy Pan and co-authors tested
SBCA makes daily allocations to small baskets of U.S. large caps, always long-only and fully invested. The two-stock group holds Caterpillar (CAT) and Gilead (GILD). The four-stock group adds GS and KO; the six-stock group also holds NVDA and MRK. Each stock supplies a headline score alongside a 30-day window of standardized log returns.
BERT-base-uncased produces the score. The authors fine-tune it once on 7,330 headline observations dated through 31 December 2018, using next-day direction as the label, then freeze it. They rescale its positive-class probability to [-1, 1]. On days without news, they fill the raw probability at a neutral 0.5, which becomes zero after rescaling. Freezing the model keeps the later text input free of look-ahead.
The fusion module applies a tanh to the text features to make a gate g. It multiplies the price features by (1 + g), allowing the headline input to change which price patterns receive emphasis. A text path is added back in before a softmax actor sets the weights. An EMA with α = 0.4 smooths those weights before trading. After a 0.25% one-way commission, the reward is log net return minus 0.1 times squared negative log return and 0.005 times turnover. SBCA combines that gated fusion with an actor-critic and a linear value head.
The data span 2012 to 2022. Training uses 2012 to 2018, validation 2019 to 2020, and testing 2021 to 2022, a test of 503 trading days. SBCA's test Sharpe is 0.8184 in the two-stock group, against 0.7870 for monthly-rebalanced equal weight. With four stocks, the comparison is 0.8108 against 0.7804.
What does the headline score add?
On 4,970 held-out observations, the encoder reaches ROC AUC of about 0.50 in both validation and test. In sample, it reaches roughly 0.7 to 0.9. The authors say the score "carries essentially no out-of-sample directional information." They ask instead whether the fusion can take a noisy input without damaging a price-based policy. Their answer depends on the critic.
The ablation crosses gated fusion with plain concatenation and the critic with plain policy gradient, producing four variants. In the two-stock group, Sharpe is 0.8013 for the plain baseline (SB), 0.8171 for critic only (SBA), 0.8092 for gate only (SBC) and 0.8184 for the full model. With four stocks, gate only scores 0.8034, below the plain baseline's 0.8048. The full model scores 0.8108, just below critic only at 0.8111.
None of the 36 ablation tests survives Benjamini-Hochberg correction. The nearest has p = 0.0017 against a critical value of about 0.0014. The authors also estimate main effects from the grid using bootstrap intervals. Only two of 54 intervals exclude zero. Both belong to the critic in the two-stock group: +0.0125 Sharpe [+0.0008, +0.0250] and +0.0285 Calmar. No cell gives the gate a main effect whose interval excludes zero.
The paper states its position plainly: "We accordingly make no claim about an independent performance contribution from the gated fusion module." The gated variants do match the concatenation variants while using 23 to 36% fewer parameters. At best, the gate learned to live with an input whose out-of-sample AUC is 0.50. In their limitations, the authors write that "the positive results of this study speak to the robustness of the fusion and training design rather than to the predictive value of news sentiment." They defend the gate as a way to absorb noise without harming the policy. The four-stock result strains that defence: without a critic, the gate scores 0.8034 against the plain baseline's 0.8048.
Against monthly 1/N
The external comparisons look stronger than the ablations. Seventeen of 60 survive correction, among them SBCA's Sharpe and Sortino advantages over equal weight in the two- and four-stock groups. With six stocks, equal weight leads on Sharpe (0.9366 vs 0.8945), annual return (0.2419 vs 0.2310) and drawdown (-0.2097 vs -0.2109). The one-sided test of whether SBCA beats it on Sharpe gives p = 0.7948. The abstract mentions only an absolute-return shortfall to Equal Weight and MVO. In the body, the authors concede that SBCA's risk profile is marginally worse than Equal Weight's; Table 2 puts Equal Weight ahead on all six metrics.
Mean-variance optimization (MVO, rolling 60-month window) ends at 1.7705 with a -0.4761 drawdown. The risk-adjusted case against it is fair. PPO, trained with SBCA's own reward and EMA, posts 0.4422, 0.3750 and 0.3700 across the three groups. Its gap from SB, which scores 0.8013 in the two-stock group, dwarfs every gap within the ablation. These figures suggest that the training setup moves Sharpe much more than the news gate.
Other results give a clue to the trades. Across five seeds, Sharpe's standard deviation is at most 0.0018. Without retraining, raising commission from 0.25% to 1.00% moves SBCA's Sharpe by -0.1007% relative in the two-stock group. Monthly equal weight moves by -0.9359%. A daily policy with about a tenth the cost sensitivity of monthly 1/N appears to trade very little.
The loss includes an entropy bonus that pulls weights toward uniform, while the EMA begins at 1/N. We did not find weight paths or turnover figures in the paper. Our inference is that SBCA stays near equal weight. The authors call it "a low-turnover policy" and credit its cost stability to the EMA and turnover penalty. They also say no ablation separates those two contributions.
Choices an implementer still faces
The theory section includes the previous allocation and t/T in the state. The framework section sends only the price window and sentiment vector to the encoders, and we did not find where the other two inputs enter.
The joint loss writes the actor term as -δ·ln R. Taken as written, its gradient passes through realized log return, scaled by the advantage, instead of through log π as in the paper's policy-gradient lemma. An implementer must decide which formulation to follow.
The notation table describes the penalty coefficients as selected from a validation set; the training details describe them as chosen for scale. The supplement is where the bootstrap's null centring is deferred. The authors also disclose that training rewards use raw weights while evaluation trades smoothed weights. Turnover uses target weights rather than drifted weights. Spread and impact are omitted.
Our run measured equal weight
We could not reproduce the experiment. Our news history starts around 2020, putting the 11-year design out of reach. We had no labeled headline set for the offline BERT fine-tuning either, so we substituted a frozen pretrained sentiment model over a shorter window. The execution departed further. We capped each stock at 1/6 in a fully invested long-only book, forcing every weight to 1/6. The cap was our error. The resulting run is an untrained equal-weight book of the six names, rebalanced daily, without a gate, critic or sentiment. It tests nothing the paper claims.
From 4 January 2023 to 1 July 2024, our book returned about 54% (33.8% CAGR), with a Sharpe of 2.62 and a -10.1% maximum drawdown. The paper reports about 52% and a Sharpe of 0.8945 for six-stock SBCA over 2021 to 2022. Strategies and markets differ, so the threefold Sharpe gap is no failed replication. The total returns also cover different lengths: about 18 months for our 54%, two years for the paper's 52%.
The window explains much of the gap. The paper's test contains the 2022 bear market. Ours covers an 18-month large-cap rally that includes NVDA, with only 374 daily rows entering annualization. The strategy substitution matters less, given that the paper's six-stock equal weight scored 0.9366, near SBCA. Our $0.004-a-share commission is likely much lower than a 25 bp one-way charge on liquid large caps. Sharpe convention may contribute too: at roughly 14% volatility, a 2% risk-free deduction could account for about 0.14.
Build monthly 1/N first. Across all three groups, the paper's learned book finishes within 0.043 Sharpe of it, and with six stocks finishes below it.
Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.