A constrained allocator should spend its marginal simulation path under the portfolio it just deployed. Huh, Kim and Jeong build a control designed to expose when that rule fails.
Their CRRA investor faces mean-reverting factors that determine expected excess returns. Constant relative risk aversion, affine mu(z), convex position limits. Optimal weights separate into the myopic Merton term and a hedging term proportional to C R^u, where R^u = D_z log F^u is the factor gradient of the reduced value factor. Homotheticity supplies the wealth curvature, gamma times Sigma, analytically. In a fifty-asset problem with three factors, only a three-dimensional field remains unknown. Accuracy matters most where that hedging field meets the binding box constraints.
The algorithm runs an outer policy loop. It simulates the deployed feedback rule, then estimates the factor-adjoint field with fixed-latent OL-BPTT. This is reverse-mode differentiation through the discretized rollout while holding the deployed action fixed. Estimation occurs at selected grid points in time and factor space, the state-time restart points called anchors. A spline, affine, quadratic or fixed-tanh ridge regression fits the field. A strictly concave box-constrained quadratic program then converts the fit into fifty weights.
Every deployment uses a damped half step, beta = 1/2 throughout, chosen from a deterministic fixed-policy PDE line profile. At the end, the authors average the pre-projection control fields, meaning the unconstrained weights before application of the box. They interpolate that average and project once. The accompanying theory moves from the finite-sample operator through a population operator to exact HJB policy improvement. A cancellation removes any need for the second Pontryagin adjoint to converge to the value Hessian.
There is no market data. Every reported number comes from a simulated model. The tests cover a 100-asset no-short Merton control, a one-factor OU problem with a box at plus or minus 2.5 and a one-dimensional HJB reference, unconstrained three- and five-factor fifty-asset problems with a matrix-Riccati reference, and a three-factor fifty-asset constrained benchmark with a factorized HJB reference. Path counts range from 2^10 to 2^17, with three outer seeds throughout. The reported metrics are policy RMSE against those references, active-set error, Jaccard overlap, KKT residual and a log-certainty-equivalent gap.
The distinction from our own work matters before any result is mistaken for ours. The paper simulates risky assets within a three-factor continuous-time model. mu(z), Sigma, the factor mean-reversion matrix and the loadings are known exactly by construction. Our own application would use a discrete-time daily approximation on a liquid US-equity universe. Neither the convergence audit nor the matched-budget comparisons can be replicated on platform data, since each depends on a reference solution available only inside the simulated model. Our implementation would require a conditional-regression or neural approximation for the factor-gradient statistic, followed by a constrained QP at each rebalance. Such an exercise would test our approximation. It would leave the paper's population-level theorem untouched.
The comparison that matters
Begin with the control. In the 100-asset no-short Merton case, the improvement map is policy independent, so re-evaluation should add nothing. It instead raises the error. The post-warm-start budget is 1.84e5 path-equivalents. Spending it in one update produces RMSE 6.004e-4. Dividing the identical budget among current-policy iterations produces 1.620e-3, while pooling it into refinement under the initial policy produces 1.089e-3.
Iteration finishes worst, and pooling beats it.
The authors state the implication plainly: iteration is not a universal variance-reduction device.
Predictable returns reverse the ordering. The one-factor model begins from a myopic policy with RMSE 5.99e-2. One update lowers it to 2.67e-3. Re-estimating the adjoint under the recovered policy reaches 6.58e-4. Using the same eight operator evaluations at the initial policy leaves error at about 2.68e-3, four times worse.
The three-factor fifty-asset constrained benchmark gives the central result. After the first half step, one current-policy re-evaluation reaches on-policy RMSE 2.775e-3. Matched pooled refinement reaches 4.453e-3. The reduction is 37.69%, with a win in all three seeds. Their log-certainty-equivalent gap agrees: 6.99e-5 against 1.72e-4, three seeds out of three. These figures all use the 1/6 broad-design mixture and no transaction costs.
Together, the Merton control and the predictable-return case make the claim falsifiable. The one-shot adjoint-to-control map appeared in the companion paper, which already performed constrained updates at a hundred assets. The authors acknowledge that history. They also describe the fifty-asset scale here as a test of policy dependence rather than action-dimension scalability. The additions are the closed loop, the population iteration's convergence analysis and a designed negative control showing when the loop should be switched off.
Three numbers, fifty weights
The scaling comes from model structure and an explicit QP. Across the non-tabular tests, the fitted adjoint fields contain only 108 and 270 coefficients for three and five factors. They decode into 1,800 and 2,700 policy coefficients. By comparison, a hypothetical 41-node tensor grid would contain 3.10e7 and 5.21e10 entries. Returned RMSE is 1.208e-3 versus 4.347e-3 one-shot at three factors. At five, it is 1.846e-3 versus 7.433e-3.
The representation experiment is the part I would retain. A smooth nonlinear warp hides the latent factors. Quadratic features have oracle-fit RMSE 5.658e-3, while the loop returns 5.852e-3, a 1.4% deterioration. Iteration cannot repair misspecification. Fixed tanh features achieve oracle-fit 4.539e-5 and return 1.264e-3, cutting one-shot error by 68.1%. The quadratic figures come from three factors, and the iteration recovers nothing beyond the basis.
How far does the theorem reach?
The exact HJB iteration has a global convergence proof for the constrained CRRA population subclass. The population OL-BPTT iteration receives that result only under an occupation-measure relative-error condition, assumed uniformly across iterates and starting states. In concrete terms, the adjoint's Hamiltonian-gradient error is averaged under the deployed policy's own path distribution. It must remain below a fraction of the policy update's size.
The audit covers a narrow slice: 180 updated-policy path banks on the one-factor benchmark, with 65,536 paths each. The largest point estimate of kappa is 0.073807. The largest per-bank 95% upper endpoint is 0.073936, compared with the half-step threshold of 0.75. On the visited sequence and listed starting states, the estimate is about a tenth of the requirement.
None of this certifies the implementation, as the paper explicitly acknowledges. Its own remark says that a fixed sampled implementation carrying persistent discretization, sampling, representation or KKT error need not converge to the optimum. Reaching a joint limit in outer iterations, paths and step size would require iteration-uniform stability, which is not established. The Limitations section gives the scope directly: "Global convergence is proved for the constrained CRRA population subclass", and "No dimension-free factor-state complexity or exact convergence of a fixed biased sampled implementation is claimed." These reservations belong to the authors. The unresolved issue is the abstract's headline sentence, which omits the tail-only exception. The code receives an a posteriori certificate instead: an update is certified when its weighted magnitude dominates the squared Hamiltonian-gradient error plus the local optimality gap.
Their audit also finds fixed-policy estimator bias. At 131,072 paths, the zero-intercept calibration slope against the corrected HJB target is 1.0621 for OL-BPTT and 1.0154 for the common-random-number finite difference. Tilted relative L2 errors are 6.66% for OL-BPTT and 1.48% for the finite difference. Additional paths reduce noise while leaving the defect. The follow-up is useful. On an exact damped HJB scaffold, the defect falls with the policy-improvement residual, giving a log-log slope of 0.932 and correlation of 0.987 on the ten theorem-covered cells of twenty-one. Fixed-policy bias therefore shrinks along the iteration, which is the setting supporting the remaining results.
One implementation choice deserves attention regardless of the proof. Averaging the pre-projection field, then projecting once after interpolation, reduces exact active-set error from about 4.85% to 0.007%. Projection at the nodes followed by interpolation creates a staircase boundary. Clamping the factor state rather than extrapolating creates a false fixed point. Under the same high-precision protocol, clamping increases uniform policy RMSE from 2.05e-3 to 7.35e-2. Table 3 reports the corrected figure from the same run at greater precision, 2.046e-3.
Coverage falls off a cliff
On-policy sampling alone fails. With zero broad anchors, core on-policy RMSE is approximately 0.244, nearly two orders of magnitude above the corrected mixtures, even though evaluation is itself on-policy. Increasing the broad stationary fraction from 1/6 to 1/2 moves on-policy RMSE from 2.955e-3 to 2.758e-3. Broad RMSE falls from 7.958e-3 to 4.708e-3. Tail-only RMSE falls from 1.084e-2 to 6.113e-3. The authors selected the 1/2 mixture by sweeping over four values with the same evaluation metrics. They disclose that selection, the per-regime choice between single-grid and Richardson-corrected estimation, and the origin of beta = 1/2.
The abstract says: "In a three-factor, fifty-asset design, current-policy re-evaluation outperforms matched pooled refinement under both evaluation laws." Both refers to on-policy and broad evaluation. A third law, tail-only, changes the result. Under the selected 50-50 design, pooled initial-policy refinement remains more accurate there in all three seeds. The paper reports this and reaches the proper conclusion: the advantage is not uniform over the enlarged tail domain. Tail-only evaluation uses the stationary factor law rescaled to 2.2 marginal standard deviations, after removing the central region max_j |z_j/sigma_j| < 1.5. Within that factor space, the box generates lower, interior and upper action regions.
The paper contains no measurement on market data, leaving nothing here to replicate. Anything we built would be a construction based on the authors' idea rather than a test of their result. Their metric is policy RMSE against a self-constructed HJB or Riccati reference. A liquid US equity universe has no equivalent reference. We have previously examined a simulated edge that weakened once its environment became realistic (/articles/an-llm-tilts-twap-by-a-basis-point-in-simulation). This gap is larger. mu(z), Sigma, the factor mean-reversion matrix and the loadings are known exactly by construction, with no transaction costs and no estimation error. We found no Sharpe, no return series and no turnover in the paper.
The compute-allocation rule is worth taking into a live constrained allocator. Re-simulate under the portfolio you deployed. Estimate the hedging field rather than training a portfolio network. First establish whether the improvement map depends on policy, since the loop costs a factor of nearly three in the Merton geometry. A matched-budget comparison with mu(z) and the loadings estimated on a trailing window instead of supplied would change my view of the broader claim. The reported edge resides entirely in a hedging term built from parameters unavailable to a real desk.