ARXAI-SST offers no evidence from manipulated financial text: every test of the attack detector uses DNS tunnelling traffic.

An attack detector beside a sentiment classifier

The premise makes sense. Trading platforms draw sentiment and market data from connected devices and feeds, giving attackers a stream they could alter to steer an automated strategy. Talamala, Aloni, Vijayalakshmi, Samantaray, Swanthana and Gandhewar propose ARXAI-SST (adversarial-resilient explainable AI for sentiment-driven stock trading) to guard against that threat.

The framework trains two components. For attack detection, it scales inputs and applies recursive feature elimination (RFE), which repeatedly removes the least useful features. A graph neural network then assigns a binary attack label, using 3 layers, hidden dimension 128 and 4 attention heads. Sentiment classification starts with FinBERT embeddings, with maximum sequence length 256 and dimension 768. These feed a 2-layer BiLSTM with attention. Training uses Adam at learning rate 0.001 for up to 50 epochs, and SHAP supplies feature attributions. How a label turns into a trade is never specified. The claim is simply that the framework produces transparent, accurate trading decisions that withstand attack.

All the data come from three Kaggle sets. For detection, the authors use an "adversarial machine learning dataset" containing 5,999 instances, 2 classes and 12 features. They describe it as "network data gained during a DNS tunnelling attack". For sentiment, they use 5,791 stock-news tweets, comprising 2,106 negative and 3,685 positive examples, plus a financial sentiment set whose size we did not find.

The detector records 97.22% accuracy against isolation forest's 96.33%, a difference of 0.89 points. Its F1 is 96.56%, compared with 89.89%. On the tweets, sentiment accuracy reaches 97.74%; GPT-4o scores 87.79% and standalone FinBERT 80.04%. The financial set yields 97.75%, a figure that appears only in the ablation table.

Where do the DNS and tweet benchmarks connect?

The paper's definition of adversarial noise is small changes to news or social posts that alter a sentiment model's output while preserving meaning. The experiments never test that threat. Network data from a DNS tunnelling attack go into the detector; unperturbed tweets go into the classifier. We found no experiment in which flagged inputs are diverted from the classifier, or in which even a single tweet is perturbed. Section 2.2 asserts that filtering takes place: "The proposed module identifies such varied inputs before sentiment analysis". Since the experiments never run that filtering, the section cannot serve as the authors' rebuttal.

The SHAP discussion exposes the disconnect. In the DNS data, mA has the greatest importance, followed by mQ, vA and sA at moderate importance. The paper interprets this as evidence that "meaningful economic indicators" drive predictions and that the model reduces "dependence on feasibly manipulated keywords". These are network-traffic features. The dataset contains neither economic indicators nor keywords.

A 97.74% result buys a tweet label

The trader gets a binary label on a tweet. The measurements stop there: no return, no Sharpe, no turnover, no cost.

The classification figures themselves are difficult to accept at face value. FinBERT alone trails the full stack by 17.7 points. In the ablation table, FinBERT first appears by name in the third row, suggesting that the earlier rows exclude it. Under that reading, "BiLSTM only (without attention)" achieves 95.93%. A plain BiLSTM would therefore outperform FinBERT by 15.89 points and GPT-4o by 8.14 on those same tweets. It is hard to reconcile those gaps with all baselines having been evaluated under a single protocol.

At the bottom of the table, the baselines raise a similar concern. Positive tweets account for 3,685 of 5,791 examples, or 63.6%. SVM achieves 64.53%, logistic regression 64.92% and random forest 65.31%. Each barely improves on always predicting positive.

Arithmetic in the detection table also fails to line up. J48 has reported precision of 90.71 and recall of 88.97, implying an F1 near 89.8. The listed F1 is 95.32. For K-means, precision is 84.73 and recall 87.06, yet F1 is 84.16, lower than both. Such figures usually reflect transcription across sources rather than a single reproduced run.

The authors describe a division into training and testing phases without giving the split ratio or error bars. We also found no statement that the split follows time order. The tweets were collected from multiple Twitter handles. A shuffled split would flatter every model in the table if the same headline appeared under several handles.

Adam contributes 0.54 points; RFE keeps all 12 features and contributes 1.31

The tweet result rises from 97.20% to 97.74% with Adam, a gain credited as 0.54 points. The corresponding gain on the financial set is 0.79. But the optimizer used for the "without Adam" run is unspecified, leaving the comparison without a counterpart.

RFE is harder to explain. According to the paper, "all 12 features were chosen". Keeping every feature leaves the models with and without RFE with identical inputs. Removing RFE nevertheless reduces accuracy by 1.31 points, from 97.22% to 95.91%.

A graph is also required for the GNN. Algorithm 1 offers only "Constructing graph G (V, E)". We did not find an account of how the 5,999 rows of tabular DNS features are made into nodes and edges. The headline detection figure therefore lacks a defined model behind it.

The deployment claim survives the backtest concession

The authors acknowledge the trading gap in their conclusion. They say the benchmark datasets may fail to capture real-world market dynamics fully, adding that "live trading assessments and financial back testing were over the scope of this study". Yet two sentences before that concession, they say the architecture "has substantial potential for real world deployment in fintech platforms". They then acknowledge that the limits "may impact the evaluation of practical deployment" and leave validation to future work in real-time IoT settings.

The concession addresses only half the problem. Deployment depends on sentiment predicting prices and on a defense stopping tampered text. The paper acknowledges that the first remains untested. The second remains equally untested, even as the conclusion describes the method as "secure." Its proposed future work covers real-time deployment without mentioning tampered text.

We could not run the framework ourselves. Our news and price data would allow a sentiment-only strategy, but we lack network telemetry and DNS attack labels. Removing the detector would leave us testing sentiment-driven prediction alone. Prior work cited by the paper, including Kim et al. [11], already makes that claim.

Two results would change my view. First, measure the accuracy drop on paraphrased or perturbed versions of the 5,791 tweets, with inputs filtered by a detector trained on perturbed text. Second, run a chronological backtest of the resulting labels, net of costs. Until those results arrive, ARXAI-SST remains a 97.74% tweet classifier alongside an unrelated 97.22% DNS detector.