Trends, problems, and solutions in causality and reinforcement learning

Grimbly, St John

Trends, problems, and solutions in causality and reinforcement learning

Thesis / Dissertation

2024

Publisher

University of Cape Town

Department

Department of Mathematics and Applied Mathematics

Faculty

Faculty of Science

Abstract

This thesis reviews, examines, and investigates the trends in the fields of causality and in reinforcement learning (RL). Theory is developed for both active research areas, with a specific focus on the overlap in underlying theory. The core argument is that the RL problem can be formulated as a causal problem, where the agent is learning causal policies that maximise return (via some causal relationship implied by the policy) and does this via selecting optimal actions (performing interventions) in the environment. Although relevant in both model-based and model-free scenarios, focus is placed on model-based modalities where one can view the various models as being causal models. It is further argued that this reformulation enables various theoretical improvements in reasoning ability for a learning agent, and does this while offering improved efficiency, interpretability, robustness, and generalisation across various learning modalities. As an application of the causal methods discussed, we also investigate whether applied causal discovery can lead to disparate impacts on sensitive subgroups. Finally, we reflect on the findings, highlight open problems, and propose future research directions.

Keywords

Mathematics and Applied Mathematics

Reference:

Collections

Masters

Full item page