Automated analysis

This analysis was drafted by our research engine and has not been checked by a human editor. It may contain errors. It separates the paper’s own results from our tests, and any figures called ours come from our own backtest.

Our automated analysisOur backtest

Does PPO-HRAP's actor earn its SPY weight?

Sharpe 0.6447 against 0.3420 for buy-and-hold on 2020 to 2022 SPY, but no result is reported for the rule alone

2026-10-03 · 7 min read · Regime-aware reinforcement learning allocation · US equity ETFs (SPY, QQQ, DIA) and cash

Reviewing: PPO-HRAP: Proximal Policy Optimization with a Hybrid Regime-Aware Policy for Risk-Controlled Trading · Duong Hien Chi Kien and Thanh Trung Huynh · Read it on arxiv

Our backtest of this idea

Our automated quick test, not the paper's

PPO-HRAP Regime-Blended Cash and Three-ETF Allocation

Backtest period 2020-01-01 to 2024-07-01 · hypothetical, net of modelled costs

Why these figures are not the paper's (2)

This is not a replication of the paper

The paper's method needs large-scale training compute, which we do not have. Any strategy we build is a substitute (typically the paper's own classical baseline), so our backtest does not test the paper's claim.

The figures below measure what we could run, not the paper's own method, so they are not evidence for or against its claim.

Our own audit found this run does not follow the paper faithfully (6)

eq. (5) state: s_t = [x_t^mkt, x_t^port] in R^13.: Retain the market/portfolio state structure but use three ETF-specific feature sets, shared VIX features and four current asset weights, giving 29 components. (invalidates: Table 5 SPY metrics; Table 6 SPY seed metrics; Table 7 separate QQQ and DIA metrics; Table 8 validation rho metrics; Section 4.5 exposure frequencies)
eq. (6) market vector: x_t^mkt = [ell_t, sma_t, rsi_14,t, bb_t, z_t^vix, ret_t^5, ret_t^20, ma_t^5,20, Delta_5 vix_t].: Calculate its ETF-specific entries separately for SPY, QQQ and DIA and include each shared VIX entry once. (invalidates: Table 5 SPY metrics; Table 6 SPY seed metrics; Table 7 separate QQQ and DIA metrics; Section 4.5 exposure frequencies)
eq. (7) portfolio vector: x_t^port = [c_t, w_(t-1), upnl_t, dd_t]; upnl_t = (NAV_t - NAV_0) / NAV_0.: Represent w_(t-1) by the three current ETF weights; retain cash weight, NAV-relative PnL and current drawdown, with weights measured at the decision close. (invalidates: Table 5 SPY metrics; Table 6 SPY seed metrics; Table 7 separate QQQ and DIA metrics)
Table 1 target w*_t: bull and not stress = 1.0; bull and stress = 0.5; mixed-trend neutral = 0.3; weak-negative-trend neutral-negative = 0.1; bear without stress = 0.0; bear and stress = -0.1; bear and crisis = -0.8.: Compute every stated signed scalar target separately and log it, then clip each to nonnegative exposure and place it in one of three equal ETF sleeves. (invalidates: Table 5 SPY metrics; Table 6 SPY seed metrics; Table 7 separate QQQ and DIA metrics; Table 8 validation rho metrics; Section 4.5 high-/low-VIX exposure and fully-long frequency)

2 further finding(s) are described in the note.

These are our findings about our own implementation, not criticisms of the paper. Read the figures below as a description of what we ran.

Jan 2020Total 42.3%Jul 2024

Sharpe: 0.89
Total Return: 42.3%
Max Drawdown: -15.6%
CAGR: 8.2%
Volatility: 10.6%
Beta vs SPY: 0.47
Trades: 1,270

What the paper reports for its own strategy

SPY, test 2020–2022, seed 42, 0.1% proportional transaction cost: total return 27.62%, annualized return 8.48%, Sharpe 0.6447, Sortino 0.8588, Calmar 0.4592, max drawdown -18.47%, turnover 51.22
SPY, test 2020–2022, mean ± std over seeds {7,11,19,23,42}, 0.1% cost: total return 0.2725 ± 0.0109, Sharpe 0.6219 ± 0.0565, Calmar 0.4508 ± 0.0499, MDD -0.1876 ± 0.0210
QQQ, test 2020–2022, single run, 0.1% cost: total return 0.4149, annualized return 0.1228, Sharpe 0.7811, Sortino 1.0697, Calmar 0.4799, MDD -0.2559, turnover 39.4215
DIA, test 2020–2022, single run, 0.1% cost: total return 0.2811, annualized return 0.0862, Sharpe 0.6623, Sortino 0.8622, Calmar 0.4838, MDD -0.1782, turnover 55.5914

PPO-HRAP's best SPY result could owe as much to a seven-row lookup table as to PPO. The paper does not report what that table earns on its own (ρ = 1). We did not train PPO; our backtest below combines the rule with a substitute policy and cannot test the paper's claim.

The rule's share of every trade

Duong and Huynh trade one ETF against cash each day, allowing weights from -1 to +1. Their method, PPO-HRAP, uses PPO with a "hybrid regime-aware policy" built around a hand-written rule. The rule takes direction from the 20-day return and the spread between the 5-day and 20-day moving averages. Two positive signs mean bull; two negative signs mean bear. A VIX z-score measured over 252 days sets the stress level, with cutoffs at 0.8 for stress and 1.8 for crisis. The resulting targets are 1.0 for a calm bull, 0.5 for a stressed bull, 0.3 for neutral, 0.1 for weakly negative, 0.0 for a calm bear, -0.1 for a stressed bear and -0.8 for a bear crisis.

The PPO actor takes 13 inputs: nine market features and four portfolio features, namely cash ratio, prior weight, unrealized PnL and drawdown. It proposes a weight, but the trade uses 0.4 times that proposal plus 0.6 times the rule's target. The reward starts with log return and also steers the actor toward the rule. It deducts a penalty for increasing drawdown, whose base coefficient of 0.05 rises with the clipped VIX z-score. Further deductions are 0.02 times the squared distance from the rule's target and 0.0015 times the absolute change in weight. The intended source of gains is lower exposure when trend breaks or VIX jumps, alongside a long position during calm advances.

Training spans 2010 to 2017; validation covers 2018 to 2019. A grid from 0.3 to 0.7 selected the 0.6 blend weight on validation, where its reported Calmar was 0.2299. The test covers 2020 to 2022, or 756 SPY days. Every method faces a 0.1% proportional cost, and PPO gets 30,000 training timesteps. The authors describe the attribution plainly: "the reported gains should be interpreted as the joint effect of the prior and the learned correction."

Did the risk control pay on SPY?

For the headline seed, 42, it did. PPO-HRAP returned 27.62% in total versus 17.60% for buy and hold. Its Sharpe was 0.6447 versus 0.3420, with maximum drawdown of 18.47% versus 34.10%. Average exposure was 0.4337 in high-VIX periods and 0.8839 in low-VIX periods, the shift the design seeks.

The seed-42 comparison gives an incomplete impression. Profit-only PPO and SAC both match buy and hold to four decimals there: total return 0.1760 and Sharpe 0.3420. Across five seeds, SAC profit has a mean Sharpe of 0.8644, Calmar of 1.4733 and drawdown of 4.69%, though its mean return is lower at 0.1978. PPO-HRAP's five-seed means are 0.6219 Sharpe, 0.4508 Calmar, 18.76% drawdown and 0.2725 return.

The authors acknowledge those results and narrow the claim to "a return-leading hybrid policy with improved drawdown control relative to passive long exposure, not as a universal winner on every risk metric." They deserve credit for saying it. The abstract and conclusion do not carry that concession through: the abstract says PPO-HRAP "remains stable" across seeds, while the conclusion claims improvements in "total return and risk-adjusted metrics over both financial and RL baselines" for the main SPY run. Their stated defence is that PPO-HRAP "has the best mean total return among the RL methods and a small return standard deviation", 0.0109. Table 6 offers another defence they leave unstated. SAC's Sharpe standard deviation is 0.2417, against PPO-HRAP's 0.0565. SAC Profit (Sharpe std 0.2417), PPO Static MDD (0.3882) and PPO Variance (0.2117) all vary sharply across seeds. PPO Static MDD records the best seed-42 drawdown at 9.51%, yet its five-seed mean reaches 35.54%, worse than buy and hold.

I would put the most weight on PPO-HRAP's tight return dispersion, a standard deviation of 0.0109. Even that does not distinguish the rule-anchored policy: profit-only PPO, which collapsed onto buy and hold, is tighter at 0.0059. With 60% of the position fixed by a deterministic rule and a reward penalty for departing from it, limited variation across seeds is close to built in.

Where is the rule-only result?

The paper defines the missing endpoint. At ρ = 1, "the learned actor is ignored and the system becomes a deterministic regime rule." The validation sweep includes 0.3, 0.4, 0.5, 0.6 and 0.7. We found no result for 1.0 in validation or test.

Calmar rises from 0.1792 at ρ = 0.3 to 0.2299 at 0.6, then reaches 0.2291 at 0.7. Drawdown is shallower at 0.7, 16.50% against 17.13%, while validation return falls from 0.0800 to 0.0767 between those points. The paper warns that "an overly strong prior can suppress learned correction," but its top two settings are nearly tied. A Calmar gap of 0.0008 cannot establish whether 0.6 is a peak or the start of a plateau extending toward 1.0.

Both actor and rule are capped at 1.0. Their blend can therefore reach full long only when each reaches 1.0. The paper says the strategy spent 54.10% of the test period at full long exposure, without defining the term. If it means a weight of exactly 1.0, the actor was pinned to the same boundary as the rule on those days. For more than half the test, it contributed no different position. The concession about joint effects is candid, yet the abstract recommends blending "learned actions with a volatility-aware regime prior," subject to caveats about turnover and single-run cross-asset evidence. A desk considering a neural network needs to see whether its 40% share pays for itself. Running the rule alone at ρ = 1 over 2020 to 2022 would require no training.

We also could not find how the 0.8 and 1.8 VIX cutoffs or the seven target levels were selected. Only ρ appears in the validation sweep. And the test includes the 2020 crash and 2022 bear market, the episodes most likely to favour VIX-driven de-risking.

The cost of moving

PPO-HRAP's SPY turnover is 51.22. The paper gives 1.0000 for buy and hold on QQQ and DIA, though the SPY table omits buy-and-hold turnover. Its per-step penalty uses |w_t - w_{t-1}|, without explaining how the reported 51.22 is aggregated. Buy and hold's 1.0 looks consistent with a sum of absolute weight changes that counts the initial allocation. Under that reading, 0.1% per unit amounts to about 5.1 points of cumulative return across three years, already included in the 27.62%. The assumption deserves scrutiny: the cost is a flat 10 bps with no slippage, and we found no borrow charge for shorts generated by the -0.8 crisis target. Doubling the cost would take roughly another five points on the same arithmetic. The 10-point advantage over SPY buy and hold would remain, with little margin to spare.

The authors call turnover "far above passive or reward-only baselines" and acknowledge that QQQ and DIA are single runs. On QQQ, PPO-HRAP earns 0.4149 with Sharpe 0.7811 versus buy and hold's 0.2306 and 0.3826, at turnover 39.42. SAC profit has the higher Calmar there, 0.5739 versus 0.4799, with a 6.09% drawdown. DIA returns are 0.2811 and 0.6623 versus 0.1468 and 0.3088, at turnover 55.59. We did not find whether ρ was selected again for those assets or transferred from SPY. The paper says its features use information available before trading, but leaves the fill price unnamed. It uses the close C_t in its features and NAV_{t+1}/NAV_t in the reward. A daily strategy trading a close-derived signal needs that timing settled.

Our substitute run cannot answer the PPO question

We could not reproduce the paper's experiment. Its claim depends on PPO training for each asset and seed, and we lack the compute to run it as specified. Instead, we ran a long-only, fully funded allocation among cash, SPY, QQQ and DIA, capping each ETF at one-third. We clipped regime targets at zero, turning the -0.1 and -0.8 bear targets into flat positions, then blended in a policy action at ρ = 0.6. This differs from the paper's single-ETF long-short strategy. It has no ρ = 1 arm either, so it says nothing about whether a learner improves on the rule.

The figures in the strip above are ours, from our backtest covering 2020-01-03 to 2024-07-01. They include commissions of $0.004 per share, subject to a $1 minimum per order. Under the closing-auction fill convention, slippage is zero. Our run made 1,270 trades over four and a half years. Its Sharpe of 0.89 exceeds the paper's 0.6447, and its maximum drawdown of 15.61% is shallower than the paper's 18.47%. Those figures measure different books and periods: our long-only, three-ETF allocation from 2020 to mid-2024 and the paper's single-ETF long-short policy from 2020 to 2022. None of our result is evidence about PPO-HRAP. Our allocation cannot short during a crisis, holds at most a third in any ETF, and includes 2023 and early 2024. Two choices in our run remain unresolved. Signal-to-fill timing was specified as next open but recorded under a close convention; a separate 0.1% notional charge may also have been applied on top of per-share commissions. Whatever their effect on the figures, those questions concern our implementation first.

A rule-only SPY Sharpe well below 0.6447 at ρ = 1 over 2020 to 2022 would make the case for the actor's 40% share. Until that result is reported, the paper has not shown that the network earns it.

Our backtest stops at 2024-07-01, and everything after that date is deliberately left untouched so the same strategy can be checked out of sample later.

How our backtest worked

The steps the code we ran actually executed, from its strategy card. Ours, not the paper's — it is one automated implementation of the idea, not the authors' own.

At each decision close, require complete ETF indicator history and a 252-day VIX window.
For each ETF, classify its 20-day return and 5/20-day moving-average spread,
then select the regime target using the VIX z-score.
Clip each signed target to [0, 1]; divide by three to form its ETF sleeve.
Assign the remaining target weight to cash.
Obtain a feasible PPO action and blend it with the target:
    intended = (1 − rho) × PPO action + rho × regime target.
Trade only where required prices and funding are available; retain realized
post-trade weights in the next state. Train PPO on chronological training
returns, select rho on 2018–2019 validation Calmar, and freeze the policy
and selected rho for evaluation.

The specification calls for decisions after close t and fills at open t+1. The recorded platform cost convention instead identifies daily-bar MOC fills; the actual fill timestamps require reconciliation before the timing description can be treated as verified.