The paper's R² 0.59 measures memory of order flow; the forecasting results give traders much less to work with. Start there if you are looking for a short-horizon price signal.
Mohl, Bismuth and their co-authors train an 8.2M-parameter Mamba-3 state-space model autoregressively on LOBSTER message streams. They use eight large caps (GOOG, AAPL, NVDA, AMZN, META, TSLA, MSFT, AMD), covering 2022-01 to 2025-12. Each submission, cancel and execution is encoded as a 26-token block. The model never sees the order book. The question comes from the Othello work: can a model shown only moves develop an internal board? Here, that means representing the book and its recent flow.
The authors inspect two locations in each of the 13 layers with kernel ridge probes. They read the residual stream, with 256 dimensions, and the recurrent state, the fixed-size memory updated at every token, with 65,536 values per layer. The probing sample is GOOG in January 2026, outside training: 2,014 windows of 500 messages across 20 trading days. Targets include the touch and order-flow imbalance (OFI, the signed change in best-quote queue sizes from Cont, Kukanov and Stoikov), alongside traded volume, realized volatility and depth. Two controls do much of the argumentative work. A random-weights twin has the same architecture; a ridge readout uses 28 features from an L10 book snapshot.
For level-1 OFI over 500 messages, the recurrent state gives R² 0.59, falling to 0.53 under forward-chaining CV. The twin manages 0.01; the boundary book snapshot gives 0.12. Reading the residual stream, the usual location for Transformer interpretability, recovers OFI, volume and volatility at R² of 0.06 or less.
Where the contribution lies
Earlier limit-order-book foundation models were assessed externally, through benchmark realism and predictive accuracy. This paper examines the inside. Its attribution result is the part I had not seen before: Mamba-3's recurrence admits an exact attention form, allowing the authors to assign credit to individual messages. The newest message supplies 58% of total state write magnitude. Yet it supplies just 7% of the decoded OFI signal, half of which requires the last 56 messages.
At the probe layer, four of eight heads have half-lives between 28 and 66 messages. No head in the random twin's corresponding layer exceeds a 10-message half-life. Training has made a flow integrator out of an architecture that does not provide one by itself.
This tells us more about the model than markets.
The OFI target covers the same 500 messages the model has just read. Decoding it reconstructs observed history. A lone snapshot cannot contain a path, so the 0.12 book baseline is weak by construction. Anyone with the reconstructed book that LOBSTER supplies alongside its messages can calculate the paper's Equation 1 exactly. I regard R² 0.59 as a lossy copy of that accumulator.
The authors make a cost argument: a linear map extracts the state with one matrix product, much more cheaply than estimating these quantities through autoregressive rollouts. That saving is relevant to someone already running the model rather than the accumulator.
Can the state beat the book on the next move?
Barely. The authors acknowledge as much in their introduction. A linear readout of the frozen state explains about 4% of the variance in the next-25-message mid move, against 0.036 for the L10 book snapshot. For future top-5 OFI, the state explains at most 5%, compared with 0.013 or less for the book. This is a small edge the book genuinely lacks. Beyond roughly 100 messages, nothing survives, and the twin remains at or below zero throughout.
The direction results are much the same. Among windows in which the mid moves, accuracy at H=25 is 61.2% for the state, 61.8% for the book and 50.5% for a majority baseline. This calculation excludes every flat window. At short horizons, those are most of the sample.
Keeping the flats in a three-class macro-F1 gives the state something to show. Its scores are 0.33, 0.36 and 0.42 at H=10, 25 and 50. The book records 0.28, 0.27 and 0.38; the twin records 0.28, 0.23 and 0.37. From H=200 onward, the book equals or beats the state.
Every horizon is measured in messages. We did not find a conversion to clock time, so these horizons cannot be interpreted as holding periods.
Steering beliefs, then generated markets
Steering means adding a direction to the state across all 13 layers. For a given feature, the direction is the mean state of top-tercile windows minus the bottom-tercile mean. The next-message response is monotone and falls outside all 30 shuffled-label null axes. Along the OFI axis, buy probability rises from 0.42 to 0.61. Pushing volume raises the probability of an execution as the next message from 0.002 to 0.090. Pushing turbulence widens the next-price forecast nine-fold.
The harder test comes when the model generates 50 messages from that steered state and replays them through a matching engine. Volume survives the replay: median executed size rises from 25 shares at zero dose to 187 at +2.4, and falls to 1 at -2.1. Steering OFI changes generated sell-side flow, while the mid remains within the null band. Steering volatility also leaves realized volatility within the null band.
The authors disclose these results and rank their control as volume, then order flow, then volatility. Their conclusion nevertheless presents the steerable state as a way to generate scenarios without retraining. By their own replay figures, only the volume scenario changes a realized market statistic; the order-flow intervention changes sell-side flow without moving the mid, and the volatility intervention moves nothing.
The tails expose the readout
Across the ten stocks, raw level-1 OFI R² ranges from 0.26 to 0.60, apart from META and MSFT, where it is near zero. The authors attribute this to concentration in the tails. Among the eight training stocks, R² declines linearly with the share of target variance in the top 1% of windows (fit R² 0.94, Spearman -0.93, p=0.002).
Winsorizing at 1% or ranking the target brings OFI L1 to at least 0.4 and top-5 to at least 0.5 for every stock, including JPM and ORCL. That repair is fair for their question: does a representation exist? It also confirms their concession that the linear readout misses extreme bursts, precisely the windows that matter most to a risk desk. The two unseen stocks fall about 0.15 below the fitted line. The paper demonstrates transfer of a representation. Transfer of a trade remains untested, with no PnL anywhere in the paper.
We could not run this ourselves. Our data ends at daily and one-minute OHLCV bars. OFI, the 500-message windows and the event-time targets require the full message stream, including queue sizes at each level.
The paper is an event-data research instrument for readers who already have LOBSTER-grade feeds. Training length is the result that could change my view: LOB-Bench plateaus about a third of the way through training, yet OFI decodability continues to rise. If a longer run lifted the 25-message forward R² clear of the book's 0.036, this would become a signal paper.