Machine Learning Models Explanations and Adversarial Attacks

Abstract

The paper considers the practical construction of adversarial attacks (evasion attacks) on machine learning models using explanations of their operation. Despite the fact that machine learning models, in general, are a black box, there are schemes for constructing explanations (their approximations) that allow us to evaluate how exactly a decision is made. Even if our model is not a decision tree, we can obtain a similar explanation for decision making in the model. One example of such schemes is the use of SHAP values. Such an approach allows us to form attacks in black box mode. If the training dataset of the attacked model or even a part of it is known, the attacker can use it to train his model of arbitrary architecture. Then, explanations can be constructed for this model, which can be used to form an attack. Since adversarial attacks are portable, such attacks can be reproduced on the attacked model. The source code for such experiments is given in this paper. Network traffic classification models in the Internet of Things system are considered as an attackable example.