Application of adaptive ensembles of machine learning methods to the problem of time series forecasting

Abstract

The purpose of this work is to study the process of applying adaptive ensembles of machine learning methods to the problem of time series forecasting, mainly for streaming data processing. For this, experiments with different types of regression algorithms were carried out, both classical methods for ensembling decision trees (random forest, gradient boosting) and neural networks, in particular convolutional and recurrent ones, were studied. For research, time series of application performance metrics, generated with the Apache Kafka service, were used. To apply machine learning algorithms to the trained data, the series prediction problem was converted to a general supervised learning problem using the sliding window technique. During the experiments, predictions were made for a long interval, for this a recursive forecasting strategy was applied. To improve the results of analysis in streaming mode, an ensemble model was proposed, which, as new data became available, based on errors, recalculate the weights that determine the contribution made to the final prediction by each individual algorithm. The possibility of initializing these weights by pre-training the model on a part of the training set was explored. During the experiments, ensemble methods showed good results in terms of accuracy, and the proposed adaptive weighted averaging technique really turned out to be able to improve the forecasting efficiency. The results of using neural networks were less impressive, which may be due to the use of a relatively small sample of data for training.