AQAI QuantAI research lab for systematic strategies

Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysisOur backtest

Neither Ridge orientation beat equal weight on 82 stocks

Safe/Follow/Reverse limited its worst shortfall against its fallback to 15.9bp; the collateral budget alone kept it to 10.6bp.

2026-11-04 · 7 min read · Online Portfolio Allocation and Benchmark-Relative Risk Control · US equities

Reviewing: Safe Portfolio Learning from Untrusted Predictions: Signed Trust, Abstention, and Cross-Universe Polarity Failure · Amirhossein MohammadpourAzari, Mohammad Zarei and MohammadAmin Fazli · Read it on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

Protected Safe/Follow/Reverse Controller with Matured-Evidence Abstention

Backtest period 2020-01-01 to 2024-07-01 · hypothetical, net of modelled costs

Why these figures are not the paper's (1)

Our own audit found this run does not follow the paper faithfully (10)

  • eq 1 capped simplex, turnover, and drift: Δ_N = {w ∈ R_+^N : 1^⊤w = 1, w_i ≤ w_max}, TO_t = ||w_t − w̃_t||_1, w̃_t = [w_(t−1) ⊙ (1+r_(t−1))]/[1+w_(t−1)^⊤r_(t−1)].: Retain the drift and simplex definitions, impose the additional platform cap, and execute explicit liquidation/purchase legs at cohort cash boundaries rather than netting one refresh displacement. (invalidates: Direct transfer of the paper's reported turnover levels and turnover savings; direct reproduction of its execution-cost path)
  • Section 2.1 one-period net return: w_t^⊤r_t − c||w_t − w̃_t||_1.: Use the declared self-financing cash-boundary entry and exit fee factors, with the same cost coefficient per traded dollar. (invalidates: Direct reproduction of original net-return tables; original fixed-path transaction-cost crossover)
  • eq 2 original paper Safe anchor: OPMD(w̃_t,g) = arg min_(w∈Δ_N){⟨g,w⟩ + (1/η_w)D_KL(w||w̃_t) + λ||w−w̃_t||_1}, w_t^Safe = OPMD(w̃_t,g_t^safe).: Use equal-weight opening targets over the annual screened universe. (invalidates: Original-anchor development CAGR and turnover savings; direct reproduction of OPMD empirical comparisons)
  • eq 4 maturity and normalized Safe-relative rewards: m_s = t_s + H − 1, ℓ_s,a = log(F_s,a/F_s,Safe), x_s,a = ℓ_s,a/B ∈ [−1,1], x_s,Safe = 0.: Maturity is the open H trading sessions after entry; use the task's reward bound and unchanged Safe-relative normalization. (invalidates: Paper session-end maturity indexing; original numerical log-unit collateral floor; direct reproduction of original cohort rewards)

6 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total 13.2%Jul 2024
Sharpe
0.13
Total Return
13.2%
Max Drawdown
-40.7%
CAGR
2.8%
Volatility
22.1%
Beta vs SPY
1.00
Trades
494,800

What the paper reports for its own strategy

  • Frozen primary strategy (Legacy Full-Ridge Reverse), 82-stock holdout, Feb 2017–Sep 2025, 5bp/unit turnover: CAGR 13.88%, ann. vol 18.28%, Sharpe 0.803, MaxDD −38.03%, avg turnover 0.965%, −0.43% annual log growth vs Equal Weight, one-sided p=0.83
  • Frozen secondary (Clean Full-Ridge Reverse), same period and cost: CAGR 13.74%, Sharpe 0.798, MaxDD −37.92%, −0.55% vs EW
  • Development Certified execution (= Safe OPMD), 24-stock, Feb 7 2017–Sep 18 2025, 5bp: CAGR 16.71%, Sharpe 0.779, MaxDD −44.49%, turnover 0.104%
  • Exploratory (post-holdout) Clean Consensus Abstention, 82-stock, 5bp: CAGR 14.17%, Sharpe 0.815, MaxDD −37.32%, turnover 0.793%, −0.18% vs EW
  • Exploratory signed controller, 82-stock, 5bp: terminal −15.9bp and worst shortfall 15.9bp relative to induced Safe benchmark, 62.0% abstention

A forecast that trails equal weight whichever way you trade it offers no alpha to manage. MohammadpourAzari, Zarei and Fazli acknowledge that directly. "The contribution is safe adaptive use of untrusted predictions, not demonstrated alpha," they write. Their proposition is a permission layer: rules governing when a forecast gets access to capital. On those terms, the paper is honest. Its turnover figures and realized 15.9bp shortfall are useful results. The unresolved question is whether the evidence gate, its most interesting layer, contributes anything beyond the simpler controls.

What gets permission to trade?

The controller starts from distrust of the prediction's sign. A signal might work in its stated direction in one universe, work in reverse elsewhere, or fail either way. Each day it therefore constructs three fully invested, long-only portfolios. Safe uses KL mirror descent, with an L1 penalty on trading, to produce a low-turnover target without consulting the signal. Follow adds a 0.10 L1 tilt in the forecast's direction. It overweights the top 20% of ranks and underweights the bottom 20%, leaving total investment unchanged. Reverse applies that tilt with the sign reversed.

Overlapping cohorts hold positions for five days. A cohort opens each session as another matures. For each candidate, the reward is cohort log wealth relative to Safe, divided by a bound of 0.05. The forecast then has four layers to pass before reaching capital.

  1. A master combines 13 Fixed-Share learners through Hedge. Each learner assumes a different switching rate; the master starts with half its mass on Safe and proposes action probabilities.
  2. The gate examines the last 50 matured rewards, requiring at least 25. An active action survives only when its mean exceeds zero by more than 1.645 standard errors. Rejected probability returns to Safe.
  3. The collateral budget reserves one unit for each unit of active probability in every unresolved cohort. Its reserve starts at 1, receives 1 more in replenishment, and has a cap of 2.
  4. The resulting allocation trades only within a 20% satellite. The remaining 80% follows Safe in the core.

The theory supplies a regret bound against switching sequences among the three actions, alongside a pathwise collateral bound. It also guarantees a floor: protected wealth stays above 80% of the Safe benchmark. Development uses a 24-stock universe, concentrated in energy, materials, mining and utilities. The frozen holdout contains 82 disjoint names across sectors. Both samples span February 2017 to September 2025; the holdout has 2,162 cohorts. The development forecast was a walk-forward Ridge fitted to 40 features and retrained 18 times. Its mean validation IC was 0.148. Trading costs are 5bp per unit of turnover.

Both signs failed on the 82-stock holdout

The frozen test was clean. Equal weight compounded 14.38%, versus 13.88% for the pre-specified Reverse orientation. Reverse lagged by 0.43% a year in log growth, with Sharpe of 0.803 against 0.813. The pre-registered one-sided bootstrap returned p = 0.83. Its interval for the gap ran from -1.33% to +0.44%. The secondary target fared worse, at -0.55%.

Reversing the choice after seeing the result offers no rescue. Against equal weight, Follow returned -0.19% on the clean target and -0.30% on the legacy target. The authors mark that comparison as exploratory. Nor had development produced much encouragement: the earlier certified-execution rule accepted no advice in 2,166 decisions and consequently reproduced Safe. Weakness in the signal ran through every stage.

Low turnover did carry over. Safe came close to equal weight's holdout Sharpe, at 0.811 versus 0.813, while trading 0.077% daily against 1.070%. It nevertheless compounded 37bp a year less.

The holdout's scope matters. Its 82 names share development's calendar window. As the authors concede, this tests transfer across asset universes "while holding the macroeconomic environment broadly aligned." The design leaves decay in a future regime untested.

The 80% floor was far from binding

Protected wealth on the post-holdout controller reached a minimum of 0.998406 relative to Safe. That amounts to a worst shortfall of 15.9bp. The certified floor permitted 2,000bp.

Most of that distance follows from arithmetic. The authors attribute it to four conservative controls stacked together: the Safe prior, gate, budget and 20% satellite. On the reference path, mean executed active probability was 0.108. That probability acted on a 0.10 tilt within a fifth of the account. Rewards were small as well. The largest absolute cohort log reward reached 0.0081, compared with the 0.05 bound. Little damage was almost assured before learning began. Even Always Reverse, allocating full active mass within the 20% satellite, suffered a worst shortfall of only 57.3bp against Safe.

The guarantee is relative. The authors make this explicit: on the audited path, Safe itself dropped to roughly 0.942 of starting wealth, and the controller offers no protection against that decline. The cumulative cohort-reward chart requires care too. It adds overlapping five-day rewards, a sum the authors describe as a learning quantity.

The theory's penalties tell different stories. Delay contributes just 2.74 in total, or 0.0013 of L1 movement per cohort, a small and genuinely informative penalty. Intervention contributes 1,131 units across 2,162 cohorts. This corresponds to mean active probability withheld of 0.52 per decision. As a numerical bound, regret is dominated by that term. The authors acknowledge that its value lies in the decomposition rather than in forecasting performance.

Does the gate earn its place?

On the reference path, it does not. The authors report the comparison themselves: the full controller had a 15.9bp worst shortfall and 62.0% abstention. Removing the gate while retaining the budget reduced shortfall to 10.6bp. Keeping the gate without the budget gave 16.5bp; removing both gave 26.8bp. Always Reverse lost as much as 57.3bp. Always Follow finished +13.4bp ahead, with an 18.2bp worst shortfall.

The authors defend the gate on the grounds that abstention is its purpose: it withholds funding from orientations unsupported by matured rewards. The supplement supplies a fair complication. With the legacy signal, the ranking flips. Full-controller shortfall is 2.8bp against 10.6bp without the gate. Across three signal panels, comparing the complete controller with budget-only gives the gate a win, a tie and a loss. Without the budget, however, the gate reduces shortfall in every panel relative to removing both layers: 16.5bp against 26.8bp, 1.7bp against 26.9bp, and 11.3bp against 26.9bp.

The complete controller makes its strongest case in the synthetic test. Without a planted signal, it abstained 97.7% of the time and lost 0.3bp. At planted strengths of 0.03, 0.06 and 0.12, it gained +26.9, +63.8 and +145.6bp over Safe. Abstention at those strengths declined to 35.4%, 33.2% and 32.0%. Always Follow lost at every strength. The generator determines the strength, though, as the authors acknowledge.

Among the exploratory variants, consensus abstention performed best. It compounded 14.17%, with Sharpe of 0.815 and turnover of 0.793%. Equal weight still led by 0.18% a year. Repricing the same fixed path puts parity near 31bp per unit turnover, using a linear model that excludes impact and spreads. At 5bp, the variant falls behind by about 21bp a year.

Our build: Sharpe of 0.13 over 2020 to 2024

We built a version using our own data. The following figures are ours, covering January 2, 2020 to July 1, 2024. We selected an annual top-100 US large-cap universe. In place of mirror descent, our Safe held equal weights of 1/100; in place of the factor-neutral Ridge, we used a 20-session price-momentum rank. Across 494,800 trades, total return was 13.23%, or roughly 2.8% annualized. Sharpe was 0.13 and max drawdown reached 40.71%.

Our return and Sharpe are far below the paper's frozen primary path, which earned 13.88% CAGR with Sharpe of 0.803. The books and windows differ. Both portfolios are long-only and close to equal weight, making each headline largely a reflection of its universe's beta over its own window. Annualized volatility in our run was 22.09%, compared with 18.28% on the paper's frozen primary path. Our period runs from 2020 to mid-2024; theirs covers February 2017 to September 2025.

The calendar difference is concrete. Our run begins directly in the 2020 crash, consistent with its drawdown. It includes 2022, while leaving out the paper's rallies from 2017 to 2019 and mid-2024 to 2025. Execution probably contributes too. We cannot quantify that drag without knowing account capital. Each sleeve refresh sells and repurchases whole baskets, producing about 437 orders a session at $0.004 a share, subject to a $1 minimum. The paper instead charges 5bp on drift-relative turnover ranging from 0.08% to 1.07% daily.

Our universe may explain some of the remaining gap because it retains distinct share classes, although we cannot establish the direction of that effect. We cannot fully account for the difference. We also produced no Safe-relative series. Our run therefore says nothing about the 15.9bp claim.

A later-period test would settle the gate

The permission layer is sensible engineering. Most protection on the clean reference path comes from the budget together with the 20% satellite: shortfall is 10.6bp with budget-only, versus 26.8bp with neither layer. Market-data evidence for the gate is mixed. Comparing the full controller with budget-only, it worsens the clean signal's result, at 15.9bp vs 10.6bp. It improves the legacy signal, at 2.8bp vs 10.6bp, and ties on the feature component, at 10.6bp vs 10.6bp.

A frozen later-period run in which the gate beats budget-only on shortfall would change my mind. The authors call for a frozen test of the complete controller on a third untouched universe, ideally under a distinct calendar regime.

Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

Initialize five independently compounded sleeves in each account:
  Safe benchmark: 1/5 each; core: 0.8/5 each; satellite: 0.2/5 each.
Initialize all learner bases at (Safe, Follow, Reverse) = (0.5, 0.25, 0.25).
Start with no pre-holdout reward evidence; initialize reserve R=1, commitments P=0.

Before each eligible opening decision:
  1. Select the annual top-100 non-ADR stock universe by capitalization.
     Retain outgoing constituents in outstanding cohorts through maturity.
  2. From prior closes, compute close[t-1]/close[t-21] - 1.
     Rank valid forecasts; give bottom/top quintiles direction mass -0.5/+0.5.
     Missing-history coordinates are zero; no ranking information means d=0.
  3. Set Safe weights to 1/100. Form Follow/Reverse = Safe +/- alpha*0.1*d.
     Choose each maximal common alpha preserving nonnegativity and both caps.
     Require the cost-inclusive ex-ante reward-bound check; otherwise use Safe.
  4. Query only strictly previously settled learner and evidence states.
     On an uninterrupted schedule, use qbar[max(0,s-H-1)].
  5. For each active action, accept only if n>=25 and
     mean(x)-1.645*sample_std(x)/sqrt(n)>0, using its last 50 matured rewards.
     Transfer rejected probability only to Safe.
  6. Set active executed mass=min(gated active mass, max(0,R-P)).
     Scale both active probabilities proportionally; transfer the rest to Safe.
     Store the decision and reserve commitment before consuming same-open rewards.
  7. Refresh sleeve ((s-1) mod 5), splitting satellite cash into action subaccounts.
     Liquidate outgoing holdings and purchase incoming baskets at observed opens.
     Hold fixed quantities for five open-to-open intervals; maintain all shadows.

At maturity, after the incoming decision is frozen:
  Compute positive net factors F_a and x_a=log(F_a/F_Safe)/0.3; x_Safe=0.
  Verify bounds without clipping; aggregate executed wealth as sum(q_exec[a]*F_a).
  Settle stored commitments, credit q_exec dot x and scheduled replenishment,
  and withdraw only uncommitted reserve above the operating cap.
  Update outer Hedge weights using pre-update base rewards, then update each
  fixed-share base and finally the gate's evidence ledger.

Never use partial rewards or same-open releases for the frozen decision.
Stop new directional exposure after a reward-bound breach; retain actual losses.
Skip incomplete executable baskets without fabricated prices or redistribution.
Open adaptive cohorts only when full maturity lies inside the test window;
otherwise use Safe, then liquidate at the final available in-window open.