A roughly 58/42 buy/sell split is enough to stop the calibrated GLFT rule from making money in this simulator. That crossover deserves attention. Moret and Lillo locate it at |p_buy - 0.50| of about 0.08 for the empirical GLFT benchmark, using expected PnL across p_buy in their zero-intelligence book. Their learned market maker remains profitable over p_buy in [0.40, 0.60], then breaks in the same fashion. Both ranges are narrow.
The threshold belongs to their calibrated generator. Its regime model draws p_buy uniform on [0.20, 0.80] at width omega = 0.30. No real-book measurement appears in the paper: the GLFT calibration data and every strategy evaluation come from the calibrated simulator, making all PnL figures below simulator output. The failure mechanism carries beyond this setup. The frontier advantage remains tied to their generator.
We did not reproduce any of it. Replication would require implementing both the Rainbow-style distributional DQN and the event-driven zero-intelligence limit order book used for training and evaluation. Our equity data consists of 1-minute OHLCV through 2024, without quotes or book depth, while their AMZN LOBSTER window falls outside it. An independently built zero-intelligence book would test our order-flow assumptions and leave their LOBSTER-calibrated environment untouched.
The simulator and the agent
The venue is an event-driven zero-intelligence limit order book of the Santa Fe type. It tracks each order under FIFO queue priority and re-centers the grid, so every fill maps to a specific resting order. Independent Poisson flows generate limit orders, market orders and cancellations. Moret and Lillo calibrate the rates from LOBSTER Level-3 data for AMZN, covering 28 trading days from 1 August to 10 September 2025 and roughly 2.9e7 messages. They discard the first and last hour of each session. The fitted rates are lambda = 0.06, mu = 0.10 and theta_cxl = 0.02 per simulated second.
GLFT receives a separate calibration inside the simulator. Passive probe orders posted at each quote distance produce a censored waiting-time estimate for exponential fill intensity. With a 0.5 second observation window, the estimated parameters are A = 0.1507 per second and kappa = 2.335 per tick. Sigma = 0.30 ticks per root-second is taken from the plateau in the volatility signature plot. Passive spread capture supplies the profits here. The practical problem is keeping persistent one-sided fills from pinning inventory at the cap.
The competing policy is a Rainbow-style distributional DQN. Its value head uses C51, with 101 atoms supported on [-3, 3], n-step 3 targets and six discrete actions for quote offsets. It decides once per simulated second, or about six book events apart. The reward combines dampened fill PnL with a quadratic inventory penalty and adds a one-sided wall beyond q_max/2. A hard cap of eight lots is imposed by dropping one leg of the selected action.
Training has three stages. Algorithm A learns under stationary symmetric flow. Algorithm B then fine-tunes under regime-switching flow, adding two state features. One is a Bayesian change-point belief about directional bias and expected run length. The other adjusts the agent's exposure on its own resting quotes for queue position. Algorithm C uses a bandit to reweight training scenarios.
Every policy runs on matched seeds in one simulator. Each policy receives the same order-flow realization through the same queue, which is the paper's main methodological answer to RL market-making studies that replay historical books with heuristic fills. A fixed arrival history cannot establish which fills another quoting policy would have obtained under consistent queue priority.
The authors are equally direct about the benchmark result. GLFT exceeds their naive at-best baseline in mean PnL only in the risk-neutral limit. Across 1000 paired episodes of 5000 LOB events, the paired median difference is +0.02 at gamma = 1e-6. It then falls to -0.015 and -0.10 as risk aversion increases, with p well under 1e-4 at the top. GLFT pays for inventory control with return.
Inventory saturation is the result
Persistent directional flow wrecks the controller trained on stationary order flow, and performance deteriorates monotonically with persistence. At a mean regime length of 15 market orders, Algorithm A earns about +30% of its no-regime PnL. By 240, it reaches -91%. Those results cover 100 paired episodes of 100,000 LOB events at phi = 1e-3.
At a mean regime length of 60, terminal PnL for Algorithm A is -46% of its stationary benchmark (std 38%). The fine-tuned policy reaches +37% of the same benchmark (std 10%). A paired t-test on terminal PnL reports p below 1e-4. Inventory traces explain the gap. Time-averaged inventory dispersion rises to 1.89x the stationary baseline for the unadapted agent and falls to 0.55x for the fine-tuned policy. The unadapted policy repeatedly reaches plus or minus eight lots, then waits for the regime to reverse.
The closed form cannot respond because its quotes are flow-blind by construction.
GLFT shows the same failure. Mean terminal PnL declines monotonically as persistence tau_r increases, turning a profitable stationary strategy into one that loses money systematically. The authors' action-space ablation supports the diagnosis and exposes a training-budget effect. Under stationary training, six actions and nine actions are effectively tied: +0.339 against +0.331 across 300 paired episodes. The three added cells are selected 10.3% of the time. During the 4000-episode non-stationary fine-tune, however, the best moving average of training terminal PnL separates to +0.097 against -0.005. Some of the apparent advantage comes from convergence under a short training budget.
Is it learning flow or its generator?
The hidden process is a piecewise-constant buy probability drawn uniform on [0.20, 0.80], with width fixed at omega = 0.30. Regime lengths are exponential on the market-order clock. In the appendix, the Bayesian filter sets its reset hazard to h = 1 - exp(-1/tau_r), using a single characteristic regime duration. The generator uses the same timescale for regime durations. The agent therefore receives a Beta-Bernoulli segment model whose switching rate is supplied, with two directional modes left to distinguish.
The stress tests reveal the dependence. If every regime draws its own persistence scale and the agent cannot observe it, mean terminal PnL declines only about 6.9% from the fixed-tau_r = 60 benchmark. This result comes from 50 paired episodes of 500,000 LOB events, and the left tail grows heavier. A second test retains correlated regime signs across boundaries with probability 0.85 while drawing imbalance intensity uniform on [0.00, 0.30]. That costs about 16.4% relative to the same benchmark. Both tests randomize persistence. In each case, the tail deteriorates more than the mean, a change that mean-PnL monitoring would overlook.
Order flow remains exogenous Poisson flow without strategic counterparties, the first of the authors' three stated limitations. We see two further absences in that model: informed counterparties and any response of flow to the maker's quotes. A front-of-queue quote cannot attract the adverse flow that would punish it, since arrivals never react to the quote. The correct direction for the quote-exposure feature therefore depends on the chosen flow model. In an earlier note on a different simulator, we examined the reverse calibration, where informed flow made the top of the queue the losing position.
The scenario bandit trains on its own exam
Algorithm C holds a pool of 256 scenarios, with 128 drawn from each of two generative families. Arms producing poor recent terminal scores receive greater sampling weight. Difficulty is tracked with an exponential moving average (eta = 0.05), while each arm has a probability cap of 0.05. The easiest 10% are refreshed every 500 bandit updates.
The results improve. Under random-persistence stress, mean terminal PnL increases 16%, while empirical standard deviation falls to about 70% of the Algorithm B level. Correlated-direction stress produces a 17% gain, with standard deviation reduced to about 76%. Both estimates use 50 paired episodes of 500,000 LOB events.
Those evaluation cases come from precisely the two families used by the bandit. Section 9 states the overlap directly: the pool contains 128 scenarios from each of the Section 8 stress families. The authors disclose this while presenting +16% and +17% as evidence that the controller withstands stress. The gain is best read as training-distribution reweighting that improved the left tail within those two laws.
We found no fee or latency assumption anywhere in the paper. PnL is denominated in simulator units, leaving the risk-return frontier as a gross comparison. The paper does show distributions, including terminal-PnL histograms, standard deviation of 10% against 38% at tau_r = 60, and reductions to 70% and 76% under the Algorithm C stress tests. A reported loss quantile would change my view, whether a 5% terminal-PnL quantile or a CVaR. So would a filter that estimates its own hazard rate rather than inheriting tau_r from the generator.