Automated Machine Learning for Predicting Trends in Time Series Data

Kouassi, Kouame

Automated Machine Learning for Predicting Trends in Time Series Data

Master Thesis

2021

Abstract

Recently, a hybrid Deep Neural Network (DNN) algorithm, TreNet was proposed for predicting trends in time series data. While TreNet was shown to have superior performance to a number of alternative approaches, the validation method used did not take into account the sequential nature of time series data. It also did not deal with model update and model stability, which are important for real-world applications. Furthermore, in the TreNet paper and previous trend prediction research, the Algorithm Selection and Hyperparameter Optimisation (ASHO) is performed manually. However, manual ASHO is expensive and often results in a sub-optimal or mediocre model because it needs extensive experimentation as well as domain specific and Machine Learning (ML) expert knowledge. This dissertation replicates TreNet experiments on the same datasets using a walk-forward validation method, which includes model update. The model is tested over multiple independent runs to evaluate model stability. TreNet, which takes in both raw point data and trend line features, is compared to vanilla DNNs and traditional ML algorithms that take in raw point data features. A recent Automated Machine Learning (AutoML) namely the hybrid Bayesian optimisation and hyperband (BOHB) framework is implemented and evaluated for ASHO. The AutoML models are then compared to the manually tuned models. The results show that in general TreNet still performs better than the vanilla DNN, but not on all datasets as reported in the original TreNet paper. On non-stationary datasets, traditional ML models outperform DNN models. The AutoML experiments found optimal configurations that produced models that surpass or compare well against the average performance and stability levels of configurations found during the experiments with manual tuning for ASHO across four datasets. This work highlights the importance of using an appropriate validation method and evaluating model stability while developing and testing ML models for time series applications. It also demonstrates that AutoML techniques such as BOHB are effective to automatically finding a well-performing models for predicting trends in time series data, thus making ML model development more systematic and less error-prone.

Keywords

Computer Science

Reference:

Collections

Masters

Full item page