No-shorting Merton RL reduces to regression and a quadratic program

The log-utility portfolio rule Li, Yu, Yuan and Zeng test comes down to regression followed by a quadratic program. Randomized actions supply the regressors; changes in log wealth supply the response. The fitted coefficients then enter an optimization over the nonnegative orthant. The authors give this regression themselves as the constant-coefficient logarithmic version of their martingale evaluation criterion. Their general evaluation procedure fits value and q coefficients using grid-sampled moment conditions, without requiring a closed-form wealth regression. The abstract advertises the QP step too: it uses fitted coefficients without taking market coefficients as inputs. This reduction follows the authors' own account.

Across two simulated two-asset markets, the learned weights finish 0.016515 and 0.025312 (Euclidean) from the constrained Merton oracle. Every figure in the paper comes from simulated constant-coefficient paths. The authors make no empirical claim about markets.

Can long-only deployment learn from short positions?

The paper uses exploratory continuous-time RL: the agent samples portfolio weights from a density. An entropy bonus rewards a more dispersed density, with temperature λ setting its strength. No-shorting applies only to the conditional mean. The average sampled portfolio must therefore be nonnegative, while individual draws remain unrestricted.

That distinction leaves the optimal portfolio density fully Gaussian. Its mean is μ̂/γ, with μ̂ minimizing ½μ'Σμ minus the excess-return term over μ ≥ 0. The covariance is λ/(γh(t)) Σ⁻¹, where h(t) solves an auxiliary ODE. Consumption follows a Gibbs density under CRRA and a Gamma density under log utility. Both cases have verification theorems.

At deployment, the agent drops the randomness and trades the mode. For a Gaussian, the mode equals the mean, making the deployed portfolio long-only. The strongest result is exact: at every positive temperature, this mode equals the classical constrained Merton portfolio. It holds under log utility and under CRRA whenever the ODE is well posed. CRRA consumption is the exception. Its mode, h(t)^(-1/γ), can differ from the classical rate at positive temperature. As temperature ε approaches zero, the gap is at most of order ε|log ε|.

Training still takes short positions.

At time zero in the interior market, λ = 0.1 gives the oracle policy a variance of 1.262626 on asset 1 around a mean of 1.515152. The standard deviation is about 1.12. Roughly 9% of those time-zero draws on asset 1 are therefore short, about one draw in eleven. The initial actor is more dispersed: precision 0.1 means variance 10 per asset. With its mean at zero, half the draws on each asset are short. The paper is explicit that the mode meets the constraint "although individual training actions may have negative components." Those draws would be short sales in a live account. Exploration has to remain in simulation.

Nor does long-only imply unlevered. The interior oracle holds (1.515152, 1.565657), roughly 308% of wealth in risky assets, with borrowing at r = 0.03 funding the position. The boundary oracle puts 166.7% into a single asset. The constraint prohibits short stock while allowing cash borrowing. A gross cap requires a sum constraint, beyond the orthant results proved here.

At 0.75 correlation, clipping gives the wrong no-shorting weights

The boundary design is the paper's best page. Both assets have positive stand-alone Sharpe ratios, 0.35 and 0.50, with correlation 0.75. The unconstrained optimum, (-0.285714, 1.809524), shorts the weaker asset. Clip that short to zero and the weights become (0, 1.809524). The constrained oracle instead holds (0, 1.666667). Because the short leg hedged asset 2's variance, removing it also calls for a 0.142857 reduction in asset 2. Re-solving the QP makes that 0.142857 cut; clipping misses it.

The learner finds the active set. Over 581 passes, learned m1 stays at zero apart from an early estimate of 0.0030. Every one of the last 100 updates chooses the oracle active set. For asset 1, the learned Karush-Kuhn-Tucker (KKT) slack is 0.004739, versus a true multiplier of 0.005. The authors describe the zero as "consistent with the complementarity conditions in Proposition 2.4, rather than being inferred from a negative estimate of the first asset's own excess return."

The constraint is strictly binding, though the multiplier is just 0.005. The simulation's b^q error, 0.001084, stays within that margin. Real data present a harder estimation problem. At 20% volatility for asset 1, three years of observations pin down its annual drift to roughly ±0.115, more than twenty times the multiplier.

The regression underneath the log-utility experiments

The log-utility implementation spells out the reduction. For each grid step, the log-wealth change divided by Δt, plus consumption, equals r + b'A − ½A'KA plus noise. The regression uses 1, A1, A2, −½A1², −A1A2 and −½A2², with a small ridge. The authors then project estimated K onto the positive-definite cone. Whatever policy generates the observations, the population coefficients are b = e and K = Σ. This allows sufficient statistics to be pooled over all 581 passes. Fitted (b, K) feed the QP that updates the portfolio. We encountered the same QP-decoder pattern in an earlier review.

Consumption is fixed in these log-utility experiments. The exact update (5.13) has no dependence on fitted parameters: it is Gamma(1 + 1/λ, g(t)/λ), where g(t) = 1 + T − t. The deployed rate, 1/g(t), consequently starts at 1/2 and reaches one at maturity. The authors identify their numerics as covering only the logarithmic specification. They give no numerical CRRA example, where a fitted h(t) determines consumption.

The chosen value-function class contains the true fixed-policy value. Its coefficients change from batch to batch, as the authors acknowledge. They also explain that those changes do not enter the portfolio update, which draws on pooled estimates.

Recovery is precise. It is also expensive. The interior errors are 0.003574 for b and 0.002262 (Frobenius) for K after 150,000 simulated one-year trajectories, each with 252 steps. The off-diagonal estimate is 0.006650, against 0.006000. Each design uses a single seed, 20260520; we found no reported dispersion across seeds. The improvement proof covers only the exact q-function. For its fitted counterpart, the authors say finite-sample monotonicity "requires a separate error analysis."

Why compute the 1.11e-5 log-utility gap analytically?

The authors evaluate deployed modes on 5,000 fresh paths using the classical log objective. There is no entropy term and no charge for costs. In the interior design, learned utility is −1.089976 against the oracle's −1.090042. The paired difference is +6.57e-5, with standard error 8.32e-5. At the boundary, the corresponding figures are −1.160688 against −1.160766, a difference of +7.89e-5 with standard error 1.64e-4.

Analytical gaps of −1.11e-5 and −4.32e-5 resolve the comparison without simulation error. The authors use them to answer the Monte Carlo noise. They describe the interior sign reversal as "compatible with Monte Carlo variation", which the arithmetic supports. Our own calculation puts the boundary difference of +7.89e-5 at about 0.5 standard errors. The same interpretation fits there.

These results also reveal how flat log utility becomes near the optimum. A mode error of 0.016515 costs 1.11e-5, about one seventh of the standard error. That flatness makes the learned policy look indistinguishable from the oracle. The market is stationary and frictionless by construction, so neither costs nor alpha decay enter the comparison.

Our ETF version stayed near cash

Our numbers need their limits attached. Daily bars require a finite-grid discretization, so this run says nothing about the continuous-time verification theorem. All exploratory draws, including shorts, remained inside bootstrapped training data. Only the nonnegative modes traded. ETF returns need not obey constant-coefficient diffusions, and the paper supplies no real-market benchmark for comparison. This was our own construction from the idea.

We used CRRA with γ = 0.5 and λ = 0.001 on SPY, QQQ, TLT and GLD. Each fold lasted 63 sessions and trained on the preceding 756 sessions through 10-session block bootstraps. Per fold, we ran three seeds, 10,000 trajectories and 21 passes, fitting on the current batch. A 0.25 cap per ETF limited total risky exposure to 100%.

From 2020-01-02 to 2024-07-01, our total return was 0.69%. Sharpe was 0.15 and volatility 1.02%. Over 3,698 trades, beta to SPY was 0.04 and maximum drawdown was −1.62%. These results are net of per-share commissions with a $1 minimum.

The differences from the paper matter. We trained on bootstrapped real history and traded held-out ETF returns, which need not meet its GBM assumptions. We used 10,000 training trajectories against 150,000, and our fits did not pool observations across batches. The exact modal-recovery identities apply to the nonnegative orthant; they do not extend to our box caps. Our consumption remained a shadow quantity inside the book, departing from the withdrawal step we had specified. Commissions on 3,698 trades reduced returns relative to the paper's frictionless evaluation. The size of that reduction depends on account size, which we did not fix.

None of those differences explains volatility of 1.02% with caps that permitted full long-only exposure. We cannot fully account for it. A fitted K large relative to β on small batches is one candidate. Rejected fits defaulting to zero means are another, as is an eligibility check set to 60 dates instead of 756. We have checked none of the three against our diagnostics. The near-cash behaviour remains unexplained.

The constrained-QP result and the lesson about clipping stand on their own. A CRRA example and a finite-sample bound for the plug-in update would make the algorithm something a desk could size.

Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.

For each nonoverlapping 63-session evaluation fold:
  End training at the preceding calendar month-end.
  Require 756 training sessions and eligible pre-window DFF observations.
  Freeze separate training and evaluation cash rates.
  Construct synchronized training-only open-to-open returns.
  Bootstrap 10-session blocks into fold-length analysis episodes.
  For each seed, initialize Gaussian portfolio and positive Gibbs actors.
  Run 21 policy passes using current-batch CRRA trajectory moments:
    sample actions before revealing the next bootstrap return;
    fit positive terminal-normalized h(t), beta(t), and full K(t);
    reject numerical failures and underidentified fits;
    update Gaussian mean through the coupled box-and-sum QP;
    update covariance to lambda * inverse(K) and Gibbs consumption law.
  Calibrate a matched classical CRRA baseline from training data only,
    converting training ACT/360 cash accrual to session-year drift.
  Freeze fitted coefficients throughout the held-out fold.
  Before each actual open:
    compute portfolio mode and consumption mode h(t)^(-1/gamma);
    specified withdrawal: W = X * (1 - exp(-c/252));
    solve post-cost target holdings and rebalance at adjusted opens;
    enforce per-name, exposure, platform-capital and solvency checks.
  Liquidate independent fold accounts at the final adjusted close.
  Compare paired complete-fold classical utility without entropy;
    maintain a separate sequential account for drawdowns.