Retrain the model before approving its explanation. Djauhari, Yudhana and Sunardi make a persuasive case for adding that step to an audit. Neither intrinsic nor post-hoc explanations wins across the board. Reliability changes with the perturbation and with the combination of model, features and explainer. The evidence is thinner on whether the TabNet reversal travels beyond this setting. That claim depends on one architecture and seed results printed two different ways.
Five model families across 20 stocks and three markets
The models forecast five-day forward realized volatility calculated from daily log returns. The sample contains 20 large caps: 10 US, 5 European and 5 Japanese. Daily observations come from Yahoo Finance and cover 2016 to 2025. Each of the five model families is fitted to two feature panels. The six signed return lags in the first panel are nearly orthogonal, with a condition number of 1.13. The second panel contains 13 technical indicators and has heavy collinearity: its condition number is 120.5 and its effective rank is 4.26. A purged walk-forward validation uses a 756-day training block, 63-day test windows and a 126-day step.
Every stock-and-arm cell must clear a "skill gate" before entering the explanation analysis: its out-of-sample MAE must beat step-5 persistence on the same rows. The explanations then face three tests. Faithfulness is measured with AOPC, the average RMSE damage caused by zeroing features in importance order, compared with a random order. Stability is assessed under three perturbations. These are input noise at 8% of each feature's SD, movement between adjacent folds, and retraining on 5 seeds over the last 3 folds. The third test measures agreement between explainers.
The strongest part of the design holds the model fixed. The authors explain a trained TabNet network twice, using its intrinsic attention mask and post-hoc Integrated Gradients (IG). IG integrates the output's gradient along a path from a baseline to the input. Checks of the model's state before and after each attribution call leave the explainer responsible for any disagreement.
What survives a TabNet retrain?
The mask beats IG under input noise on every matched stock. On price lags, Spearman stability is 0.967 for the mask and 0.897 for IG (n=16). On technicals, the corresponding figures are 0.987 and 0.963 (n=13). Retraining reverses the ranking. Table 11 reports mask seed agreement of 0.006 versus IG's 0.077 on price lags (p=0.044). On technicals, it reports 0.086 versus 0.216 (p=0.001). Temporal shift leaves the two indistinguishable (p>0.10).
Architecture has the larger effect. Temporal rank agreement differs by 0.519 between IG on the LSTM and IG on TabNet (n=11, p=0.014). Both TabNet explanations are close to zero on price lags: 0.054 for the mask and 0.111 for IG. The GAM reaches 0.644. In Japan, the mask's temporal agreement on price lags is −0.020, a mild reversal of its importance ordering from one quarter to the next.
The authors acknowledge that the mask-versus-IG reversal is established on TabNet alone. Their conclusion leaves its reach beyond sparse attention unresolved. I agree, and the size of the reversal also remains unsettled. Table 10 gives mask seed agreement as 0.044 and 0.104; Table 11 gives 0.006 and 0.086. For IG, one table reports 0.120 and 0.265, the other 0.077 and 0.216.
The discrepancy extends to predictions. The prose reports TabNet's seed prediction correlation as 0.27 and 0.25, against 0.302 and 0.305 in Table 10. The discussion supplies a third value, 0.047. According to the text, all 20 stocks enter the seed contrast. Yet the retraining rows in Table 11 show n=16 and n=13. The tables appear to use different stock sets without disclosing the difference. That could account for the gap between Table 10 and Table 11, though it contradicts the claim that the seed contrast includes all 20 stocks.
Every version preserves the direction: the mask falls below IG.
The distance between them remains unclear.
Step-5 persistence sets a low hurdle
The gate passes 165 of 240 cells (68.8%) on a point estimate alone, as the authors acknowledge. A requirement of q≤0.05 would pass 6% of cells and remove both same-model comparisons. At q≤0.10, all four shared-model arms remain. This stricter tier admits 112 cells. Its arm rankings correlate at 1.000 with those under the primary gate for temporal agreement, rank-biased overlap and input stability. As a defence of the screen for ranking explainers, that is fair.
The best median RelMAE belongs to HAR-RV at 0.844. RelMAE divides MAE by step-5 persistence MAE; below 1 means the model beats persistence. The spline GAM on price lags is tied with HAR at 0.858 (HAR wins 11 of 20, p=0.648). HAR beats TabNet on price lags for all 20 stocks.
A training-window mean achieves a median of 0.892, matching XGBoost on price lags. On the mean measure, however, XGBoost's 0.892 beats the training mean's 0.917. The LSTM on technicals misses the gate on all 20 stocks and records a median of 1.356. The authors spell out the handicap: they deliberately exclude realized-variance lags from the price-lag panel. Gate rates also differ sharply across markets, at 49.2% in the US and 90.0% in Japan.
Ablation faithfulness and the top-5 overlap trap
Faithfulness yields no signal. In 10 of the 11 admitted arms, AOPC less its random control lies between −0.00014 and +0.00012. The remaining arm goes the wrong way. For the GAM on technicals, AOPC is 0.00551, against 0.01190 for a random order. The authors disclose the null result plainly. They also flag that roughly 51 of the 148 significant pairwise comparisons belong to the AOPC family.
Top-5 overlap creates another problem. With six features, two top-5 sets must share at least four names. Jaccard therefore has only two possible values, 0.667 or 1.000. Its random expectation is 0.7222 in that panel, compared with 0.2537 among 13 features.
Table 12 compares the TabNet mask with IG on the same network, using admitted cells. Raw top-5 Jaccard is 0.854 versus 0.549, making the smaller panel look better. Measures invariant to panel size reverse the result: Spearman is 0.464 versus 0.633, and RBO excess over chance is 0.330 versus 0.381. XOM makes the distortion concrete. Its Spearman is −0.143 alongside a top-5 Jaccard of 0.667. The authors identified this artifact in their own earlier finding. Publishing the correction deserves credit.
Put retraining into risk-model sign-off
The practical output is an audit procedure: retrain with fresh seeds, then compare agreement in explanations with agreement in predictions. For the EBM on price lags, prediction agreement is 0.712, while explanation agreement is only 0.458. An input-noise test on a single fitted model would miss that gap entirely.
Stability alone proves no truth. The LSTM on technicals never beats persistence. Its IG attributions nevertheless agree across seeds at 0.493, while prediction agreement is just 0.190.
We could not rerun the study because we lack European and Japanese single-stock histories. Restricting a rerun to the US would remove 10 of the 20 names. It would also sacrifice the cross-market check, whose rank correlations of 0.88 to 0.99 do much of the work in the paper's defence.
A second same-model contrast would change my view on the paradigm question. The authors propose Shapley attribution on the fitted GAM and EBM. That test requires no new data.