Kim and Lim find a thin signal for a brokerage's next target-price decision, with no test of stock returns. Their ten LLM scores add about two to three points of AUC after accounting for the last revision and sector. The forecast concerns what the broker will do next to its own target.

From reports to scores

The sample contains 41,640 single-stock reports covering 1,807 US-listed companies, all from one global brokerage between January 2017 and August 2026. The authors keep each PDF's title, summary and body paragraphs. Tables, figures and disclaimers are removed.

A language model masks company names and tickers, along with target-price amounts and language announcing a raise, cut or hold. It also removes upside percentages derived from the target. Qwen3.5-122B-A10B then answers ten fixed questions in a single call at temperature zero. The questions cover EPS growth, revenue growth, profitability, valuation, earnings versus consensus, shareholder returns, competitive position, industry environment, regulation and catalysts. Each answer consists of a score from -3 to +3 and a line of supporting evidence. Together these form QOS, the authors' question-based outlook scores.

The prompt distinguishes an unmentioned topic from a neutral view. Null means the report says nothing; zero means it explicitly expresses an in-line view. Roughly 65% of reports omit shareholder returns, compared with 3% for catalysts. The models nevertheless replace nulls with zero. According to the authors, missingness flags and carry-forward gave similar results.

The rationale draws on work by Xiao and Zang cited in the paper: analysts' language can move ahead of their numbers. An analyst might predict that next quarter will beat consensus before changing the target price.

What is being forecast?

The outcome is the direction of the next observed revision within the same brokerage-stock series. The authors model up versus unchanged and down versus unchanged. Unchanged targets dominate both tasks, accounting for 62.6% of current targets and 55.2% of next targets. The median interval back to the previous target is 27 days.

Past revisions already carry information. A raise is followed by another raise 37.4% of the time, compared with 23.4% after an unchanged target. A cut follows a cut 31.6% of the time, against 18.7%. Text has to improve on that persistence. The authors therefore control for the current revision as a continuous variable.

Testing proceeds walk-forward. Each rolling 36-month window contains 24 months for fitting and 12 months of purged validation for tuning. The model is then refitted and scored on the following quarter. The exercise produces 27 quarterly folds, from 2020Q1 through 2026Q3.

A small gain, with stronger competitors

With current revision alone, mean quarterly AUC is 0.557 for up and 0.551 for down. GICS sector dummies raise those figures to 0.621 and 0.579. QOS brings them to 0.640 and 0.612.

Sector contributes 0.064 on the up side; the ten scores add 0.019. For down, the scores contribute more: 0.033 against sector's 0.028. These increments depend on entry order, since sector enters before QOS. The authors make the attribution explicit: "the gain over current revision alone should not be attributed entirely to the text scores."

QOS beats the cheap sentiment measures. Loughran-McDonald records 0.627 / 0.588, while FinBERT reaches 0.622 / 0.597. Dense embeddings do better than QOS, with BGE-M3 at 0.658 / 0.630 and FinLang at 0.656 / 0.623.

Compactness is the paper's defence. The conclusion acknowledges the embeddings' higher AUCs, then argues that "the ten scores provide incremental predictive information in a compact representation." Another passage says the scores "retain much of the predictive discrimination" of embeddings. Judged by AUC levels, that is fair: QOS is within 0.018 of BGE-M3 in each direction (0.640 vs 0.658, 0.612 vs 0.630).

We use a tougher comparison: the share of the embedding's gain over controls recovered by ten numbers. BGE-M3 adds 0.037 for up and 0.051 for down over those controls. QOS captures about 51% and 65% of those gains. By that yardstick, "much of" is a poor description of the up-side result.

We did not find confidence intervals or a test of the AUC differences across the 27 folds. The significance of the 0.013 up-side gap between QOS and Loughran-McDonald (0.640 vs 0.627) therefore remains unknown.

Same-report classification exposes a weakness. Fully masked QOS distinguishes up from down at 0.859, near BGE-M3's 0.869. The distance widens when unchanged targets are the comparison: 0.718 versus 0.876 for up, and 0.734 versus 0.890 for down. The scores distinguish raises from cuts well. They have more trouble separating a change from no change, the more common case.

Evidence a reader can inspect

The supporting lines carry the interpretability claim. A reader can inspect the reason for a +2 on Q5, the earnings-versus-consensus question. That reason still needs checking. The authors leave validation of supporting evidence to future work and acknowledge that another LLM could assign different scores.

The masking comparison leaves room for doubt. Up/down AUC barely moves, from 0.860 unmasked to 0.859 fully masked. This comparison covers only same-report classification. The authors interpret it as evidence that aggregate discrimination does not depend strongly on the removed identifiers, while allowing that individual scores may change. Other forecasts remain in the text, alongside valuation multiples and narrative. Part of the reason for masking is to limit knowledge acquired during pretraining. An unchanged AUC cannot distinguish an absence of leakage from leakage through those retained forecasts and narrative.

For readers of reports, the question-level coefficients are the most useful results. Earnings versus consensus (Q5) predicts both directions, with +0.16 for next up and -0.15 for next down. Valuation (Q4) has the reverse sign to what one might expect. An undervalued assessment carries -0.27 on the next raise, conditional on the other nine scores. The authors suggest that calling a stock undervalued can justify retaining an existing target without increasing it.

Revenue growth has a +0.26 coefficient on current cuts, opposite to the sign for EPS growth. The authors identify these as full-sample regressions, separate from the walk-forward models. Ten scores enter the same logit and are adjusted for one another. We would not trade any individual sign.

Sector results vary sharply. Separately fitted sector models give pooled down-side AUCs of 0.496 in Financials and 0.506 in Real Estate, compared with 0.631 in Consumer Staples. The authors suggest questions tailored to sectors, including net interest margin and cap rates.

First, get the archive

We could not test this.

The method requires the full narrative text of a brokerage's reports, plus report-linked target-price series that retain reports with unchanged targets. Those unchanged observations represent 62.6% of current targets and 55.2% of next targets, the reference class for each task. News feeds reporting raises and cuts provide neither the full narratives nor that unchanged-target history.

The paper tests no stock returns. There is consequently no Sharpe or cost analysis. Returns are excluded deliberately: the authors want to measure the analyst's expressed outlook, whereas returns reflect much else. A trader still needs a link from the broker's next decision to a stock price. The evidence here supports a narrower application.

The industry-environment score illustrates it. Its monthly average dropped from -0.43 in 2019 to -1.25 in April 2020, before recovering to +0.41 in 2021. For the media industry, the March-June 2020 average was 1.68 points below its 2019 average. This gives a sell-side mood dashboard with citations attached. It would become a signal if someone showed that the 0.019 and 0.033 AUC gains survive a fold-level test and carry into returns.