Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems
| dc.contributor.advisor | Bassett, Bruce | |
| dc.contributor.advisor | Clark, Allan | |
| dc.contributor.author | Hayes, Max Nieuwoudt | |
| dc.date.accessioned | 2022-01-27T07:03:23Z | |
| dc.date.available | 2022-01-27T07:03:23Z | |
| dc.date.issued | 2021 | |
| dc.date.updated | 2022-01-26T13:42:32Z | |
| dc.description.abstract | Since reinforcement learning algorithms have to fully solve a task in order to evaluate a set of hyperparameter values, conventional hyperparameter tuning methods can be highly sample inefficient and computationally expensive. Many widely used reinforcement learning architectures originate from scientific papers which include optimal hyperparameter values in the publications themselves, but do not indicate how the hyperparameter values were found. To address the issues related to hyperparameter tuning, three different experiments were investigated. In the first two experiments, Bayesian Optimisation and random search are compared. In the third and final experiment, the hyperparameter values found in second experiment are used to solve a more difficult reinforcement learning task, effectively performing hyperparameter transfer learning (later referred to as meta-transfer learning). The results from experiment 1 showed that there are certain scenarios in which Bayesian Optimisation outperforms random search for hyperparameter tuning, while the results of experiment 2 show that as more hyperparameters are simultaneously tuned, Bayesian Optimisation consistently finds better hyperparameter values than random search. However, BO took more than twice the amount of time to find these hyperparameter values than random search. Results from the third and final experiment indicate that hyperparameter values learned while tuning hyperparameters for a relatively easy to solve reinforcement learning task (Task A), can be used to solve a more complex task (Task B). With the available computing power for this thesis, hyperparameter optimisation was possible on the tasks in experiment 1 and experiment 2. This was not possible on the task in experiment 3, due to limited computing resources and the increased complexity of the reinforcement learning task in experiment 3, making the transfer of hyperparameters from one task (Task A) to the more difficult task (Task B) highly beneficial for solving the more computationally expensive task. The purpose of this work is to explore the effectiveness of Bayesian Optimisation as a hyperparameter tuning algorithm on the reinforcement learning algorithm NEAT's hyperparemters. An additional goal of this work is the experimental use of hyperparameter value transfer between reinforcement learning tasks, referred to in this work as Meta-Transfer Learning. This is introduced and discussed in greater detail in the Introduction chapter. All code used for this work is available in the repository: • https://github.com/maaxnaax/MSc_code | |
| dc.identifier.apacitation | Hayes, M. N. (2021). <i>Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems</i>. (). ,Faculty of Science ,Department of Statistical Sciences. Retrieved from http://hdl.handle.net/11427/35595 | en_ZA |
| dc.identifier.chicagocitation | Hayes, Max Nieuwoudt. <i>"Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems."</i> ., ,Faculty of Science ,Department of Statistical Sciences, 2021. http://hdl.handle.net/11427/35595 | en_ZA |
| dc.identifier.citation | Hayes, M.N. 2021. Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems. . ,Faculty of Science ,Department of Statistical Sciences. http://hdl.handle.net/11427/35595 | en_ZA |
| dc.identifier.ris | TY - Master Thesis AU - Hayes, Max Nieuwoudt AB - Since reinforcement learning algorithms have to fully solve a task in order to evaluate a set of hyperparameter values, conventional hyperparameter tuning methods can be highly sample inefficient and computationally expensive. Many widely used reinforcement learning architectures originate from scientific papers which include optimal hyperparameter values in the publications themselves, but do not indicate how the hyperparameter values were found. To address the issues related to hyperparameter tuning, three different experiments were investigated. In the first two experiments, Bayesian Optimisation and random search are compared. In the third and final experiment, the hyperparameter values found in second experiment are used to solve a more difficult reinforcement learning task, effectively performing hyperparameter transfer learning (later referred to as meta-transfer learning). The results from experiment 1 showed that there are certain scenarios in which Bayesian Optimisation outperforms random search for hyperparameter tuning, while the results of experiment 2 show that as more hyperparameters are simultaneously tuned, Bayesian Optimisation consistently finds better hyperparameter values than random search. However, BO took more than twice the amount of time to find these hyperparameter values than random search. Results from the third and final experiment indicate that hyperparameter values learned while tuning hyperparameters for a relatively easy to solve reinforcement learning task (Task A), can be used to solve a more complex task (Task B). With the available computing power for this thesis, hyperparameter optimisation was possible on the tasks in experiment 1 and experiment 2. This was not possible on the task in experiment 3, due to limited computing resources and the increased complexity of the reinforcement learning task in experiment 3, making the transfer of hyperparameters from one task (Task A) to the more difficult task (Task B) highly beneficial for solving the more computationally expensive task. The purpose of this work is to explore the effectiveness of Bayesian Optimisation as a hyperparameter tuning algorithm on the reinforcement learning algorithm NEAT's hyperparemters. An additional goal of this work is the experimental use of hyperparameter value transfer between reinforcement learning tasks, referred to in this work as Meta-Transfer Learning. This is introduced and discussed in greater detail in the Introduction chapter. All code used for this work is available in the repository: • https://github.com/maaxnaax/MSc_code DA - 2021_ DB - OpenUCT DP - University of Cape Town KW - Advance Analytics LK - https://open.uct.ac.za PY - 2021 T1 - Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems TI - Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems UR - http://hdl.handle.net/11427/35595 ER - | en_ZA |
| dc.identifier.uri | http://hdl.handle.net/11427/35595 | |
| dc.identifier.vancouvercitation | Hayes MN. Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems. []. ,Faculty of Science ,Department of Statistical Sciences, 2021 [cited yyyy month dd]. Available from: http://hdl.handle.net/11427/35595 | en_ZA |
| dc.language.rfc3066 | eng | |
| dc.publisher.department | Department of Statistical Sciences | |
| dc.publisher.faculty | Faculty of Science | |
| dc.subject | Advance Analytics | |
| dc.title | Optimising the Optimiser: Meta NeuroEvolution for Artificial Intelligence Problems | |
| dc.type | Master Thesis | |
| dc.type.qualificationlevel | Masters | |
| dc.type.qualificationlevel | MSc |