Analysis of Machine Learning Algorithms for Time Series Prediction

Naidoo, Kimendree

Analysis of Machine Learning Algorithms for Time Series Prediction

Master Thesis

2021

Abstract

Due to the rapidly increasing prominence of Artificial Intelligence in the last decade and the advancements in technology such as processing power and data storage, there has been increased interest in applying machine learning algorithms to time series prediction problems. There are many machine learning algorithms that can be used for time series prediction problems but selecting an algorithm can be challenging due to algorithms not being suitable to all types of datasets. This research investigates and evaluates machine learning algorithms that can be used for time series prediction. Experiments were carried out using the Artificial Neural Network (ANN), Support Vector Regressor (SVR) and Long Short-Term Memory (LSTM) algorithms on eight datasets. An empirical analysis was carried out by applying each machine learning algorithm to the selected datasets. A critical comparison of the algorithm performance was carried out using the Mean Absolute Error (MAE), the Mean Squared Error (MSE), the Root Mean Squared Error (RMSE) and the Mean Absolute Scaled Error (MASE). The second experiment focused on evaluating the stability and robustness of the optimal models identified in the first experiment. The key dataset characteristics identified; were the dataset size, stationarity, trend and seasonality. It was found that the LSTM performed the best for majority of the datasets, due to the algorithm's ability to deal with sequential dependency. The performance of the ANN and SVR were similar for datasets with trend and seasonality, while the LSTM overall proved superior to the aforementioned algorithms. The LSTM outperformed the ANN and SVR due to its ability to handle temporal dependency. However, due to its stochastic nature, the LSTM and ANN algorithms can have poor stability and robustness. In this regard, the LSTM was found to be a more robust algorithm than the ANN and SVR.