A live trading desk cannot treat an untested drift threshold as a control. Both computable elements in this paper remain hypotheses, a status the authors acknowledge in their Limitations section: "The adversarial-input protocol (Section 3.6) and the systemic-layer simulation (Section 4.4) remain illustrative rather than fully empirically populated. The taxonomy and ARS are a triage instrument, not a validated risk metric." Their answer appears in the same paragraph. Readers should avoid treating the case-study magnitudes as calibrated estimates for other strategy classes, "though the qualitative mechanisms (opaque vendor-model dependency, undisclosed vendor-side updates, and shared-feed crowding) generalize directly." General mechanisms justify constructing a monitor. A trigger requires data. Without that threshold, the monitor has yet to become a control.
Three parts of the paper lie outside any backtest, and they need stating before our figures appear. Reward-function approvals, vendor version attestations, escalation logs and kill-switch procedures exist as internal records. Simulation cannot create them. Our data also cannot measure systemic crowding across other institutions. At best, we can form proxies from our exposures, volume, volatility and cross-sectional factor concentration. We also lack the proprietary embeddings behind MSRR. In their place, we used in-code NLP over available news text, press releases and earnings transcripts. The resulting signal is an implementable variant. It does not contain the paper's governance apparatus.
Aldridge and Krawciw have a stronger argument than their evidence can carry. SR 11-7 style model risk management assumes that a validated model returns consistent outputs for consistent inputs between review cycles. Production retraining breaks that assumption by construction. Pre-, intra- and post-trade rules engines still restrict the system's permitted actions. They cannot establish whether the decision rule producing orders matches the approved rule. Reward hacking, in the sense used by Amodei and co-authors and cited by the paper, can occur wholly inside those limits. The governance gap exists. The authors describe their framework as the first to gather these strands into one computable architecture designed for finance.
Four layers make up their response. The policy layer handles the reward function as a risk policy document, complete with version control and sign-off. The engineering layer adds a drift statistic and a kill switch tied to the model's internal confidence rather than its self-reported confidence. Layer 2.5 is the composition layer, where the entire pipeline becomes the unit of governance instead of an individual model. A systemic layer addresses crowding. Above them is the Agentic Risk Score, an ordinal triage number built from four dimensions: Autonomy, Reversibility, Blast Radius and Policy Mutability. Each receives a score from 1 to 3. The weights sum to one, and ARS at or above 2.5 acts as a provisional trigger for mandatory Layer 2 controls.
The architecture outruns the evidence. The authors supply no market data of their own. Initial support for the governance gap comes from an informal LinkedIn poll conducted in May 2026. Among respondents, 88% report having no operational governance framework for agentic AI, while 0% report being unaware of agentic AI deployment. Its sample is self-selected, and the paper never gives its size. Next comes a hand count of Form ADV filings. AI use is disclosed by 75 of 100 of the largest U.S. money managers, while 24 of those 75 (32%) disclose a formal governance policy.
Both numerical exhibits are entirely synthetic and come from the authors' own code. The two case studies rely on third parties. One is the MSRR strategy from Didisheim and co-authors, a long-short U.S. equity portfolio trading the residual component of LLM news embeddings. The other reconstructs the July 2026 collapse of Situational Awareness LP from press reports. That discretionary hedge fund ran roughly 4x leverage and concentrated its positions in the AI infrastructure trade.
Can the eleven-period alarm handle a live book?
The drift monitor deserves attention for one reason. Observed costs and decisions are enough to calculate it, with no view into the policy's internal state. Its basis is a regret decomposition from Aldridge's own earlier work. Cumulative regret is expressed as the sum of per-period covariances between realized costs and decisions, plus a bias term. A firm using a vendor model faces precisely this setup: outputs can be watched and costs paid, while the weights remain hidden.
The paper shows the statistic detecting a change point deliberately embedded in a constructed series. Across T = 500 periods, cost shocks follow a standard normal distribution. Before t = 300, decisions are independent of contemporaneous costs. From t = 300 onward, the policy sizes positions using the realized cost draw, with gamma = 1.2 and decision noise carrying a standard deviation of 0.5. The trailing 20-period sample covariance averages 0.124 before the change and 1.154 afterward, producing a ratio of 9.3x. A trigger is placed at the validated-regime mean plus four standard deviations, level 0.913. It first sounds at t = 311.
One path. Eleven periods. A change inserted by design.
The known onset informs every choice among W = 20, k = 4 and gamma = 1.2. We did not find a false-alarm rate, a power curve, or a comparison with either rolling correlation or a CUSUM on P&L. Nor does the 11-period latency have a distribution estimated over repeated draws. The authors call the exhibit a calibrated illustration. They also say an adopting institution can re-sweep the parameters against real cost and decision data. Fair enough. The defensible conclusion remains tight: when drift is this strong, a monitor fitted to it sounds eleven periods late. Its false positive rate on a real portfolio remains unknown, even though that rate will determine whether users keep the alarm active.
The regret-covariance identity underneath the monitor has genuine force and clear limits. We examined its derivative theory in an earlier note. For linear policies, the gradient with respect to the intercept vanishes, leaving the objective unable to rank budget-feasible intercepts. Optimization sets a higher hurdle than detection because this monitor only requires the covariance to shift. Even so, implementers should remember that the tracked object is a signed sizing rule.
The route to 39.2% and 79.3%
The abstract highlights a calibrated crowding model in which joint drawdown probability climbs from 39.2% to 79.3%. The authors solve backwards to both targets. They select (mu, beta) within a two-agent shared-factor model so that the analytic bivariate-normal joint-breach probability reaches those chosen figures. A million-draw Monte Carlo then confirms 39.1% and 79.2%. The paper is explicit on this point, describing the exercise as a transparent reconstruction of the reported figures rather than an attempt to recover an unpublished methodology.
Two imposed levers produce the increase. Marginal breach probability is fixed by construction at 0.60 in calm conditions and 0.85 under stress. The solved regime mean shift is mu = -0.253 in calm and mu = -1.036 in stress. Much of the change in level therefore precedes any movement in beta. Shared-factor loading increases from 0.460 to 0.901, while implied return correlation rises from 0.212 to 0.812.
The parameter sweep contains the useful information. Figure 3 moves beta continuously while holding the stress-regime mean shift fixed, and joint-loss probability accelerates as crowding intensity increases. By contrast, the 39.2 to 79.3 pairing illustrates arithmetic selected by the authors.
MSRR's rolling five-year Sharpe supplies the live evidence for crowding. Across the sample it ranges roughly 2.1 to 4.5. Following 2018, when transformer embeddings became widely available, the measure slips from peaks near 4.5 into a range of 2.1 to 2.5. Didisheim and co-authors ascribe the deterioration to wider incorporation of news information. Competitor positioning is never observed along that chain. In the series, crowding cannot be disentangled from alpha decay and data-vintage effects. Layer 3 needs an observable measure before it can function as a control, perhaps provider concentration across funds or factor-overlap statistics. Requiring disclosure of the identity and provider for every third-party news or embedding feed is a sensible first field. Collecting that field still does not measure beta.
Why the composition layer belongs
MSRR records an out-of-sample Sharpe of 3.1 from 1996 to 2022, roughly twice the strongest individual JKP anomaly. It begins with an initial 24-month window and refits monthly. Turnover reaches 75%, above every JKP factor. Scale turns the strategy into a governance concern. A 110-million-parameter BERT encoder produces Sharpe 1.5, compared with 4.1 from a 405-billion-parameter Llama3. Meanwhile, point-in-time and full-foresight versions of a chronologically consistent LLM perform indistinguishably.
Together, those findings make a silent vendor model upgrade a first-order change in the return distribution. A training-cutoff attestation reveals little about that shift. The direct response is to record the vendor model identifier and version for every production inference, which the authors include as a fifth Layer 2.5 requirement. The paper leaves the main operational issue untested: whether vendors will provide inference-level attestations in the first place. Its 90-day roadmap also assigns no price to the controls on its schedule.
The authors publish an awkward result for their own instrument. Situational Awareness LP receives A=1 and M=1, the minimum scores for autonomy and mutability, alongside B=3 for a roughly $45bn portfolio operating at leverage up to 4x. MSRR receives A=1-2, R=2, B=2-3, M=2. An ARS weighted toward autonomy and policy mutability consequently treats the fund that lost most of its AUM within weeks as safer than a monthly-refit equity signal. The authors acknowledge this in the text and use it to show that the four dimensions cannot substitute for one another.
Our substitute fell 45% in five years and ten months
The proprietary MSRR embeddings were unavailable to us. We therefore built the signal with in-code NLP applied to available news text, press releases and earnings transcripts. From 2020-01-01 through 2025-10-08, the variant returned -45.14% in total. Its Sharpe was -1.47, Sortino -1.96 and Calmar -0.20, with a maximum drawdown of -49.68% and annualised volatility of 5.35%.
The outcome is dreadful.
The universe comprised the top 500 U.S. stocks by point-in-time capitalization, excluding ADRs. For each symbol, we formed a daily text score and residualized it cross-sectionally against log market cap, PE, PS, PB and EV/EBITDA before z-scoring. The portfolio bought the top 20% and sold the bottom 20%, using equal dollars and zero net exposure. Normal conditions carried a gross target of 1.0 and a 10% single-name cap, with execution at the close. Text records ended at the close of the prior trading day.
A governance overlay added a daily drift statistic. On a breach, it paused new entries and halved the gross target until 20 consecutive clean days had passed. We used daily bars and charged four tenths of a cent per share, subject to a $1 order minimum and a 1bp commission cap. Slippage beyond commissions was set to zero, a choice that favours the result.
Our gate and the paper's statistic observe different objects. The worked example measures covariance between realized costs and a signed sizing rule using gamma = 1.2. Our version observes nonnegative turnover, defined as the sum of absolute weight changes. Accordingly, the movement from 0.124 to 1.154 and its 9.3x ratio concern a different quantity from the one monitored by our overlay. Our gate can also sound mechanically because of the cost proxy, without detecting anything comparable to reward hacking.
The obvious return comparison is unavailable in the desired form. Our daily 500-name portfolio produced a Sharpe of -1.47 over five years and ten months, using in-code sentiment. Didisheim and co-authors give MSRR an out-of-sample Sharpe of 3.1 across 1996 to 2022, based on monthly refits and proprietary LLM embeddings. The figures describe different setups. Their differences all favour the reported MSRR comparison: twenty-seven years against six, monthly refits against daily rebalancing, and LLM embeddings against keyword-level sentiment. The paper supplies no strategy performance of its own.
This was one automated pass. Weakness in our variant would first implicate our text source and residualization. None of the paper's argument depends on our figures, and our result is no verdict on the authors' work.
What belongs on the desk now?
Begin with an inventory of mutable models and vendor dependencies. By my reading, this mainly consumes calendar time, and the paper's first 30 days describe the task fairly. Hard de-risking should fire without waiting for a human. The Situational Awareness case supports that specific claim: deleveraging came from a prime broker's margin desk instead of the fund's risk process, leaving the fund with a worse price. Vendor model identifiers should also be recorded on production inferences. The move from Sharpe 1.5 to 4.1 across model scale puts those identifiers inside the return-risk process.
The regret-covariance monitor and crowding score should remain hypotheses carrying a calibration cost. Neither is applied to real cost and decision data anywhere in the paper. A narrow, testable result would change my view of the drift statistic: a false-alarm rate from a live strategy's cost and decision series, accompanied by a comparison with a CUSUM on P&L showing that the covariance term detects something missed by the simpler test. Until that evidence is published, the eleven periods belongs to the simulation.
Several internal cross-references in the paper are broken. The survey section is directed to two different numbers, and the crowding simulation to two more. The fault is minor, yet it makes a proposed architecture built around auditability harder to audit.