Anomaly detection with autoencoders

D.A. Safronov, Y.D. Kazer, K.S. Zaytsev

Abstract


The purpose of this work is to reduce the cost of troubleshooting digital equipment by improving anomaly recognition methods based on the use of autoencoders. To do this, the authors propose to use two types of autoencoders: a deep feed-forward autoencoder and a deep convolutional autoencoder, and as a comparison, it is proposed to use two supervised machine learning methods: the logistic regression method and the support vector machine. The comparison confirmed the effectiveness of the proposed autoencoders. The NSL-KDD dataset was chosen for experiments with the algorithms. It includes more than 10,000 measurements and 41 parameters characterizing the network flow. This dataset contains both normal and abnormal network stream data. Before training machine learning algorithms and autoencoders, the current data set was pre-processed: correlation elimination, categorical data binarization, data grouping into attack categories. The results of using autoencoders developed by the authors to search for anomalies have shown the effectiveness.

Full Text:

PDF (Russian)

References


Almeida A. et al. The complementarity of a diverse range of deep learning features extracted from video content for video recommendation // Expert Systems with Applications. Pergamon, 2022. Vol. 192. P. 116335.

Hammouche R. et al. Gabor filter bank with deep autoencoder based face recognition system // Expert Systems with Applications. Pergamon, 2022. Vol. 197. P. 116743.

Qu C. et al. Predictive anomaly detection for marine diesel engine based on echo state network and autoencoder // Energy Reports. Elsevier Ltd, 2022. Vol. 8. P. 998–1003.

Ma Q. et al. A novel model for anomaly detection in network traffic based on kernel support vector machine // Computers and Security. Elsevier Ltd, 2021. Vol. 104.

Wang H. et al. Anomaly detection for hydropower turbine unit based on variational modal decomposition and deep autoencoder // Energy Reports. Elsevier Ltd, 2021. Vol. 7. P. 938–946.

Zhao W. et al. On the use of artificial neural networks for condition monitoring of pump-turbines with extended operation // Measurement: Journal of the International Measurement Confederation. Elsevier B.V., 2020. Vol. 163.

Egusquiza M. et al. Advanced condition monitoring of Pelton turbines // Measurement. Elsevier, 2018. Vol. 119. P. 46–55.

Protić D. Review of KDD Cup ’99, NSL-KDD and Kyoto 2006+ datasets // Vojnotehnicki glasnik. Centre for Evaluation in Education and Science (CEON/CEES), 2018. Vol. 66, № 3. P. 580–596.


Refbacks

  • There are currently no refbacks.


Abava  Absolutech Convergent 2022

ISSN: 2307-8162