Fine-tuned language models find no return edge in RSI and ADX on large caps. Kumar tests nine models ranging from 0.8B to 32B parameters; their accuracy on 2025 return questions runs from 0.370 to 0.383. A table of train-period base rates gets 0.379. Kumar is explicit: "Our claim is deliberately not alpha; it is calibration." Whether that calibration supports the proposed deployment is the question left to settle.
Nine base models, three corpora
Kumar calls these "System-1" decision models: small language models that read a state and a typed question, then produce probabilities over single-token answer codes in one forward pass. They handle three question types. One is multiple choice, another an ordinal score, and the last a yes/no proposition, called noul in the paper, returning P(yes). Across nine base models from the Qwen and K2-Horizon families, Kumar trains LoRA adapters with a fixed recipe: rank 16, learning rate 1e-4, two epochs.
The training material comes from three corpora. General competence is measured on a public decision benchmark with 14,576 training rows. A synthetic policy corpus has 2,651 rows, with stated threshold rules and labels calculated by code. For market outcomes, the data cover 40 liquid US equities, daily bars from 2016 and seven technical features. Questions ask about horizons of 1, 5, 10 and 21 sessions. Realized returns supply the labels: direction, or one of five bins with cuts at −2.23/−0.22/+1.15/+3.73%.
The splits advance through time. Training ends in 2023 with 15,640 rows; calibration uses 2024 (2,000), and development uses 2025 (2,000). The sealed 2026 test contains 1,360 rows and ships unopened. After calibration, probabilities determine dispatch: approve at P≥0.90, reject at P≤0.25, and route the rest to review.
We could not backtest this. We found neither a position rule nor a cost assumption in the paper, and access to the adapters and corpora requires a request. Without trained weights, we have no 2025 forecasts to trade against our own daily equity data.
Can the models beat a base-rate table?
For direction, the hit rate has to clear always-up before trading costs enter. The adapters achieve 0.501 to 0.521; always-up scores 0.519. The classical controls in Kumar's paper fare much the same. Ridge logistic regression on the seven features reaches 0.514. Its Brier is 0.2510 versus 0.2496 for a constant base rate, and every standardized weight is at or below 0.08. Gradient boosting gets 0.496. Costs and capacity have nothing here to erode.
Calibration presents a closer contest, which Kumar reports fairly. Against the train prior table, the 2B specialist improves accuracy (0.383 vs 0.379), Brier (0.666 vs 0.673) and NLL (negative log-likelihood, 1.206 vs 1.224). Pooled ECE (expected calibration error) favors the table, 0.019 against 0.026. A date-block bootstrap puts the Brier edge at +0.005, with a 95% interval of [−0.001, +0.010]. Kumar calls the result unresolved. That judgement holds. On the specialist's score questions, ECE reaches 0.050.
The prior table assigns a base rate to each question type and horizon. The best adapter matches it without a detectable improvement.
The adapter presented as deployable does worse than the specialist, as Kumar himself reports. For the 2B mixed adapter, Brier is 0.673 and NLL is 1.223, level with the table. ECE reaches 0.044, more than twice the table's 0.019. Accuracy of 0.376 also trails the table's 0.379. Yet the abstract describes this as "near-specialist outcome calibration". The specialist has ECE of 0.026.
A year later, calibration drifts further at 9B to 32B than at 2B
The larger models lose more calibration a year out. Outcome ECE traces a U-shape across model size, from 0.026 at 2B to 0.043 to 0.044 from 9B to 32B. Training had a 21-session up-rate of 0.555; in 2025 it was 0.482. At that horizon, the 9B model assigns mean P(up) of 0.670.
"Temperature cannot repair a shifted center," Kumar writes. His proposed remedy is rolling recalibration of the marginals inside the gate. The remedy has its own weakness: re-estimating the prior table over one year produces worse results on every 2025 metric than using the 2016 to 2023 table. Kumar recommends the longest stable window, an awkward fit with "rolling". All of this rests on one development year. Even directional accuracy has a ±3.7pp half-width across 695 noul rows. Kumar says overlapping windows make that an understatement of uncertainty.
Narrow fine-tuning costs up to 45pp of general accuracy, and mixing repairs it
The stronger result lies outside markets. Fine-tuning an adapter on a single finance corpus damages its general decision competence. Outcome-only training loses 45.3pp on the public benchmark at Qwen 0.8B; the loss falls to 14.5pp at 27B. For policy-only training, the losses are 23.3pp and 20.8pp at the two smallest rungs, then 8.7pp at 27B. The outcome corpus contains about 8x the rows and causes 1.7 to 3.8x more damage (mean about 2.3x).
The greatest damage comes from the corpus with no detectable conditional signal. Kumar treats the link with size as an association and identifies confounds, including label noise and step count.
Plain concatenation repairs the loss. At Qwen-2B, mixed adapters score 0.814 on the public benchmark versus the generalist's 0.808. At K2-3.7B, the comparison is 0.834 against 0.825. Policy accuracy stays at 0.995 and 1.000, while foreign-panel accuracy drops 0.9 and 1.1pp. The three-seed replication at 2B preserves the pattern: generalist 0.812, policy-only 0.627, outcome-only 0.461, mixed 0.813. The evidence has limits. Mixing was tested at only two sizes, and seed replication covered one.
What do 129 synthetic rows establish about a pre-trade gate?
The pre-trade gate supplies the deployment exhibit. Of 129 policy questions, the mixed adapter assigns 27 rows to P≥0.90 and 102 to P≤0.25. All three adapters make zero errors, including the generalist, and review rates range from 0 to 0.8%. Kumar supplies the one-sided 95% error bounds: 10.5% and 2.9%. A desk would reject a 10.5% bound on approvals.
Kumar agrees that adoption requires validating the bound on more data. He describes the gate as a simulation on programmatic-gold structures, although the abstract still promises "sharp, dispatchable bands". He also reports that the generalist passes this same gate without ever training on policy data. The policy specialist thus sacrifices 17.5pp of general competence for a gate the generalist already clears. Across all policy question types, the generalist still scores 0.751 against the specialist's 0.992. What the gate establishes is that the models can follow a stated threshold rule.
For outcome questions, the bands behave as an honest model should. The 2B specialist leaves 99.9% of noul mass in [0.40, 0.60); mixed adapters leave 83 to 88%, and the 9B specialist leaves 57%. Sending that mass to review makes sense. A lookup table assigning a base rate to each question type and horizon would do the same, without training costs.
We did not find how or when the 40 names were chosen, a concern for any later trading version. I would change my view if the sealed 2026 test showed a specialist's Brier advantage over the prior table with an interval excluding zero.