OpenUCT :: Browsing by Author "Shock, Jonathan"

Browsing by Author "Shock, Jonathan"

Now showing 1 - 20 of 20

Open Access
Active Inference in Multi-Objective Dynamic Environments
(2022) Hodson, Rowan; Shock, Jonathan; Smith, Ryan; Solms, Mark
Artificial Intelligence holds the promise of not only creating intelligent entities, but also unlocking the mysteries of our brains, and the nature of the subjective consciousness that accompanies them. Many paradigms of artificial intelligence are attempting to push the boundaries of the field, in order to catch a glimpse of the secrets behind general intelligence and the nature of the human mind. A less explored, yet promising paradigm is that of Active Inference - a theory which details a first-principled explanation of how agents use action and perception to successfully operate within an external environment. Much work has been done to explore the framework's viability in modelling scenarios both related to neural process theory and more classical agent-based machine learning. However, due to the relative recency of the theory, there are still many areas of comparison and evaluation to explore. This dissertation aims to investigate Active Inference's algorithmic capacity to solve more complex decision-based environments. Specifically, with varying degrees of complexity, I make use of a dynamic environment with a multi-objective reward function to investigate the Active Inference agent's ability to learn and plan while balancing exploration and exploitation, and compare this to other Bayesian Machine Learning algorithms. In doing so, I investigate some novel approaches and additions to Active Inference's algorithmic structure which include a dynamic preference distribution, a two-tiered hierarchical approach to the state space (using model-free Reinforcement Learning to solve the lower level), and the introduction of the Propagated Parameter Belief Search algorithm - a modification to Active Inference which allows the agent to perform more complex counterfactual reasoning.
Open Access
Approximating a wavelet kernel using a quantum computer
(2023) Rughubar, Rivan; Shock, Jonathan; Dietel Thomas
Machine learning and quantum computing are both fields which have gained a significant amount of popularity and attention in recent years. The intersection of these two fields, quantum machine learning, looks at whether quantum computers can aid or improve classical machine learning methods, or whether quantum computers can perform machine learning tasks which classical computers cannot. In this thesis we explore different implementations of quantum machine learning algorithms on near term quantum computers, and the limits of these systems. We focus on support vector machines and kernel methods, which are a form of supervised machine learning. We examine whether using quantum kernels to search for a quantum advantage over classical computers is suitable, and why it may be wise to search for quantum advantages using other methods. Lastly, we construct a quantum circuit which can approximate a wavelet kernel with a mean squared error over sample plots of 9.09 × 10−9 , by estimating the Fourier coefficients of the kernel. We hope that this can be used as a starting point for performing wavelet analysis on quantum computers.
Open Access
Biologically motivated reinforcement learning in spiking neural networks
(2022) Rance, Dean; Shock, Jonathan
I consider the problem of Reinforcement Learning (RL) in a biologically feasible neural network model, as a proxy for investigating RL in the brain itself. Recent research has demonstrated that synaptic plasticity in the higher regions of the brain (such as the cortex and striatum) depends on neuromodulatory signals which encode, amongst other things, a response to reward from the environment. I consider which forms of synaptic plasticity rules might arise under the guidance of an Evolutionary Algorithm (EA), when an agent is tasked with making decisions in response to noisy stimuli (perceptual decision making). By proposing a general framework which captures many proposed biologically feasible phenomenological synaptic plasticity rules, including classical SpikeTime-Dependent Plasticity (STDP) rules and the triplet rules, and rate-based rules such as Oja's Rule and BCM rules, as well as their reward-modulated extensions (such as Reward-Modulated Spike-Time-Dependent Plasticity (R-STDP)), I allow a general biologically feasible neural network the ability to evolve the rules best suited for learning to solve perceptual decision-making tasks.
Open Access
Chaos and Scrambling in Quantum Small Worlds
(2020) Hartmann, Jean-Gabriel Keiser; Murugan, Jeffrey; Shock, Jonathan
In this thesis, we introduce a novel class of many-body quantum system, which we term ‘quantum small worlds'. These are strongly-interacting systems that interpolate between completely ordered (nearest-neighbour, next-to-nearest-neighbour etc.) and completely random interactions. They are systems of quantum spin particles in which the network topology is given by the Watts-Strogatz model of network theory. As such, they furnish a novel laboratory for studying quantum systems transitioning between integrable and non-integrable behaviour. Our motivation is to understand how the dynamics of the system are affected by this transition, particularly with regards to the ability of the system to scramble (quantum) information, and potential emergence of chaotic behaviour. Our work begins with a review of the relevant literature regarding algebraic graph theory and quantum chaos. Next, we introduce the model by starting from a well understood integrable system, a spin- 1 2 Heisenberg, or Ising, chain. We then inject a small number of long-range interactions and study its ability to scramble quantum information using two primary devices: the out-of-time-order correlator (OTOC) and the spectral form factor (SFF). We find that the system shows increasingly rapid scrambling as its interactions become progressively more random, with no evidence of quantum chaos as diagnosed by either of these devices.
Open Access
Computational analysis techniques using fast radio bursts to probe astrophysics
(2021) Platts, Emma; Weltman, Amanda; Shock, Jonathan
This thesis focuses on Fast Radio Bursts (FRBs) and presents computational techniques that can be used to understand these enigmatic events and the Universe around them. Chapter 1 provides a theoretical overview of FRBs; providing a foundation for the chapters that follow. Chapter 2 details current understandings by providing a review of FRB properties and progenitor theories. In Chapter 3, we implement non-parametric techniques to measure the elusive baryonic halo of the Milky Way. We show that even with a limited data set, FRBs and an appropriate set of statistical tools can provide reasonable constraints on the dispersion measure of the Milky Way halo. Further, we expect that a modest increase in data (from fewer than 100 FRB detections to over 1000) will significantly tighten constraints, demonstrating that the technique we present may offer a valuable complement to other analyses in the near future. In Chapter 4, we study the fine time-frequency structure of the most famous FRB: FRB 121102. Here, we use autocorrelation functions to maximise the structure of 11 pulses detected with the MeerKAT radio telescope. The study is motivated by the low time-resolution of MeerKAT data, which presents a challenge to more traditional techniques. The burst profiles that are unveiled offer unique insight into the local environment of the FRB, including a possible deviation from the expected cold plasma dispersion relationship. The pulse features and their possible physical mechanisms are critically discussed in a bid to uncover the nature and origin of these transients.
Open Access
Computational Psychiatry - Neuropsychological Bayesian reinforcement learning
(2022) Wolpe, Zach; Shock, Jonathan; Cowley, Benjamin; Clark, Allan
Cognitive science draws inspiration from a myriad of disciplines, and has become increasingly reliant on computational methods. In particular, theories of learning, operant conditioning and decision making have shown a natural synergy with statistical learning algorithms. This offers a unique opportunity to derive novel insight into the conditioning process by leveraging computational ideas. Specifically, ideas from Bayesian Inference and Reinforcement Learning. In this thesis, we examine the statistical properties of associative learning under uncertainty. We conducted a neuropsychological experiment on over 100 human subjects to measure a suite of executive functions. The primary experimental task (Card Sorting) gauges one's ability to learn, via inference, the structure of some latent pattern that drives the decision making process. We were able to successfully predict the subjects' behaviour in this task by fitting a Bayesian Reinforcement Learning model, alluding to the mechanics of the latent biological decision generating process and executive functions. Primarily, we detail the relationship between working memory capacity and associative learning.
Open Access
Entanglement entropy, the Ryu-Takayanagi prescription, and conformal maps
(2017) Grant-Stuart, Alastair; Murugan, Jeffrey; Shock, Jonathan
We define and explore the concepts underpinning the Ryu-Takayanagi prescription for entanglement entropy in a holographic theory. We begin by constructing entanglement entropy in finite-dimensional quantum systems, and defining the boundary at infinity of a bulk spacetime. This is sufficient for a naïve application of the Ryu-Takayanagi prescription to some simple examples; nonetheless, we review the general theory of minimal submanifolds in Riemannian ambient manifolds in order to better characterise the objects involved in the prescription. Finally, we explore the symmetries of the boundary theory to which the prescription applies, and thereby extend the aforementioned examples. Throughout, emphasis is placed on making explicit the mathematical structures that are taken for granted in the research literature.
Open Access
Evaluating transformers as memory systems in reinforcement learning
(2021) Makkink, Thomas; Shock, Jonathan; Pretorius, Arnu
Memory is an important component of effective learning systems and is crucial in non-Markovian as well as partially observable environments. In recent years, Long Short-Term Memory (LSTM) networks have been the dominant mechanism for providing memory in reinforcement learning, however, the success of transformers in natural language processing tasks has highlighted a promising and viable alternative. Memory in reinforcement learning is particularly difficult as rewards are often sparse and distributed over many time steps. Early research into transformers as memory mechanisms for reinforcement learning indicated that the canonical model is not suitable, and that additional gated recurrent units and architectural modifications are necessary to stabilize these models. Several additional improvements to the canonical model have further extended its capabilities, such as increasing the attention span, dynamically selecting the number of per-symbol processing steps and accelerating convergence. It remains unclear, however, whether combining these improvements could provide meaningful performance gains overall. This dissertation examines several extensions to the canonical Transformer as memory mechanisms in reinforcement learning and empirically studies their combination, which we term the Integrated Transformer. Our findings support prior work that suggests gating variants of the Transformer architecture may outperform LSTMs as memory networks in reinforcement learning. However, our results indicate that while gated variants of the Transformer architecture may be able to model dependencies over a longer temporal horizon, these models do not necessarily outperform LSTMs when tasked with retaining increasing quantities of information.
Open Access
Evaluation of Plasmodium falciparum gametocyte detection in different patient material
(BioMed Central Ltd, 2013) Kast, Katharina; Berens-Riha, Nicole; Zeynudin, Ahmed; Abduselam, Nuredin; Eshetu, Teferi; Loscher, Thomas; Wieser, Andreas; Shock, Jonathan; Pritsch, Michael
BACKGROUND:For future eradication strategies of malaria it is important to control the transmission of gametocytes from humans to the anopheline vector which causes the spread of the disease. Sensitive, non-invasive methods to detect gametocytes under field conditions can play a role in monitoring transmission potential. METHODS: Microscopically Plasmodium falciparum-positive patients from Jimma, Ethiopia donated finger-prick blood, venous blood, saliva, oral mucosa and urine samples that were spotted on filter paper or swabs. All samples were taken and stored under equal, standardized conditions. RNA was extracted from the filter paper and detected by real-time QT-NASBA. Pfs16-mRNA and Pfs25-mRNA were measured with a time to positivity to detect gametocyte specific mRNA in different gametocyte stages. They were compared to 18S-rRNA, which is expressed in all parasite stages. Results were quantified via a known dilution series of artificial RNA copies. RESULTS: Ninety-six samples of 16 uncomplicated malaria patients were investigated. 10 (66.7%) of the slides showed gametocyte densities between 0.3-2.9 gametocytes/mul. For all RNA-targets, molecular detection in blood samples was most sensitive; finger-prick sampling required significantly smaller amounts of blood than venous blood collection. Detection of asexual 18S-rRNA in saliva and urine showed sensitivities of 80 and 67%, respectively. Non-invasive methods to count gametocytes proved insensitive. Pfs16-mRNA was detectable in 20% of urine samples, sensitivities for other materials were lower. Pfs25-mRNA was not detectable in any sample. CONCLUSIONS: The sensitivity of non-invasively collected material such as urine, saliva or mucosa seems unsuitable for the detection of gametocyte-specific mRNA. Sensitivity in asymptomatic carriers might be generally even lower. Finger-prick testing revealed the highest absolute count of RNA copies per muL, especially for Pfs25-mRNA copies. The method proved to be the most effective and should preferably be applied in future transmission control and eradication plans. A rapid test for gametocyte targets would simplify efforts.
Open Access
Finite system size effects on the effective coupling in scalar quantum field theory
(2024) Du Plessis, Jean; Shock, Jonathan; Horowitz William
Findings from the Large Hadron Collider (LHC) provide evidence that quark-gluon plasma (QGP) formation , traditionally associated with large nucleus-nucleus collisions, also emerges in high multiplicity proton-proton (p + p) and proton-nucleus (p + A) collisions. It has also been shown that azimuthal correlations of low transverse momentum outgoing particles from these high multiplicity p + p and p + A collisions can be modeled well using nearly inviscid relativistic hydrodynamics. One would expect the equation of state (EOS) used in such hydrodynamic simulations to receive significant contributions in systems of size characteristic of p+p collisions. Indeed it has been shown that free scalar thermal field theory receives ∼ 40% corrections to the usual thermodynamic quantities when confined using Dirichlet boundary conditions with characteristic lengths set by central p + p collisions. Furthermore, the potential importance of asymmetric system sizes has also been highlighted in quenched lattice QCD calculations using periodic boundary conditions. In order to analyse the finite system size corrections to the QCD EOS, we need to explore the finite system size corrections to the running of the QCD coupling. This thesis provides a first step toward such a calculation. We consider massive scalar ϕ 4 theory in a spacetime with periodic boundary conditions. We allow characteristic lengths to be asymmetric, or even infinite. We first recount the infinite volume NLO 2 → 2 scattering amplitude calculation using dimensional regularisation, and then introduce and employ denominator regularisation. We then perform the non-trivial calculation of the NLO 2 → 2 scattering amplitude in our compactified spacetime. This requires the derivation of an new analytic continuation of the generalised Epstein zeta function after employing denominator regularisation in order to isolate the UV divergence. Denominator regularisation is necessary, since the usual techniques in dimensional regularisation no longer applies when we allow asymmetric characteristic lengths. We confirm that taking the limit of infinite characteristic lengths yields the usual infinite space-time result. We then perform another non-trivial self-consistency check by verifying that the NLO 2 → 2 Page ii scattering amplitude satisfies unitarity in the form of the optical theorem, regardless of the number of compactified dimensions. In order to show this we derived a generalisation of a formula originally proposed by Hardy and Ramanujan, and interpret its analytic continuation in the context of Abel summation. We numerically explore the scattering amplitude in some special cases, but find the s-channel of the 2 compactified dimension case to be numerically ill-behaved. We then derive and employ a dispersion relation in order to numerically explore the s-channel. Using the Callan-Symanzik equation to derive the beta function, we find it insensitive to finite system effects, which only modifies the IR. We then perform a geometric resummation of bubble diagrams, and show that the running coupling from the beta function agrees to a leading log in energy to the resummed amplitude. Interpreting the modulus of the resummed amplitude as an effective coupling in analogy to QED, we numerically explore its dependence on energy scale as well as the length scale of the system. We find that surprisingly at small length scales the effective coupling decreases, even though the beta function is positive. This thesis has overlap with related work by the author.
Open Access
From statistical mechanics to machine learning: effective models for neural activity
(2022) Schonfeldt , Abram; Rohwer, Christian; Shock, Jonathan
In the retina, the activity of ganglion cells, which feed information through the optic nerve to the rest of the brain, is all that our brain will ever know about the visual world. The interactions between many neurons are essential to processing visual information and a growing body of evidence suggests that the activity of populations of retinal ganglion cells cannot be understood from knowledge of the individual cells alone. Modelling the probability of which cells in a population will fire or remain silent at any moment in time is a difficult problem because of the exponentially many possible states that can arise, many of which we will never even observe in finite recordings of retinal activity. To model this activity, maximum entropy models have been proposed which provide probabilistic descriptions over all possible states but can be fitted using relatively few well-sampled statistics. Maximum entropy models have the appealing property of being the least biased explanation of the available information, in the sense that they maximise the information theoretic entropy. We investigate this use of maximum entropy models and examine the population sizes and constraints that they require in order to learn nontrivial insights from finite data. Going beyond maximum entropy models, we investigate autoencoders, which provide computationally efficient means of simplifying the activity of retinal ganglion cells.
Open Access
High-school students' productive struggles during the simplification of trigonometrical expressions and the proving of trigonometrical identities
(2023) Sayster, Anthony; Shock, Jonathan; Mhakure Duncan
This study is an investigation into school students' productive struggles in the simplification of trigonometric expressions and proving of trigonometric identities. Although studies have been published on the teaching and learning of trigonometrical concepts in schools and teacher education, there is a lack of published research into students' productive struggles in the simplification of trigonometric expressions and proving of trigonometric identities. To fill this gap in the literature, this study used a sample of 16- and 17-year-old students at a rural high school in South Carolina in the United States of America to conduct a study on the use of productive struggles in the simplification of trigonometric expressions and the proving of trigonometric identities. The study used the Anthropological Theory of the Didactic by Chevallard (1992) as the main theoretical framework. However, this main framework was supported by other frameworks. The Anthropological Theory of the Didactic contends that mathematical activities such as simplifying trigonometric expressions and proving trigonometric identities must be interpreted as a human activity rather than seeing these mathematical activities as a language, the creation of concepts (for example, “simplification” or “proof”) or a cognitive process. A praxeology consists of two parts, namely (in Greek) the praxis, or “know how”, and the logos, or “know why”. The praxis is commonly known as the practical block, and the logos the theoretical block. This means that the Anthropological Theory of the Didactic can be used to describe how certain actions regarding the simplification of trigonometric expressions and proving of trigonometric identities take place, and why these actions take place. The exercises in the activities were obtained and adapted from the students' prescribed textbook. These activity questions were sequenced using the Development Cognitive Abilities Test (DCAT). The DCAT reflects Bloom's (1956) hierarchy of cognitive abilities. This means that the exercises were organised in three groups of increasing complexity, i.e., easy, medium, and difficult. The easy exercises related to the DCAT's Basic Cognitive Abilities, referred to as DCAT 1; the medium exercises related to Application Abilities, referred to as DCAT 2; and the difficult exercises related to Critical Thinking Abilities, referred to as DCAT 3. The data in this study consists of video recordings from classroom observations in real time transcribed verbatim, documentary analysis of students' assessments, and audio-recorded focus group interviews. The focus group interviews were also transcribed verbatim. Each transcription focused on a different aspect of the students' productive struggles in the simplification of trigonometric expression and the proving of trigonometric identities. Errors made by the students in written assessments were analysed using the Newman Error Analysis framework. By using Newman Error Analysis, this study could investigate and compare how the errors on the assessments were related to the students' struggles as observed during the teaching and learning of the activity questions. Due to Co-Vid 19 restrictions that resulted in logistical difficulties, only one class of 15 students participated in this study. After listening to the focus group recordings numerous times and reading the transcripts, common patterns were noted that had emerged, either from paraphrasing or from direct quotes. The primary research question is: What is the nature of the productive struggles experienced by high-school students during the simplification of trigonometric expressions and proving of trigonometric identities and how do these productive struggles influence the learning and teaching of trigonometry? The study findings were that the students struggled with “carrying out known mathematical processes” such as manipulating equations, knowing under what conditions cancellation of terms can be applied, adding and subtracting algebraic fractions involving trigonometric expressions, and factorisation of trigonometric expressions. In addition, there were misconceptions about the concept of “simplification”. Delayed impasse struggles occurred; this is when a student does not initially struggle to get started with a question, but the struggling becomes apparent as the student progresses with the question. The students committed fewer Newman errors in proving trigonometric identities than in the simplification of trigonometric expressions. Subsequently, students performed better at proving trigonometric identities than at simplifying trigonometric expressions. It could well be that through productive struggles, the students developed some of their own strategies from the simplification of trigonometric expressions. Alternatively, proving identities could be seen as “easier”, since the students already know what the “answer” should be. Nonetheless, students still struggled to carry out common mathematical processes such as factorisation and the manipulation of algebraic fractions. Regarding factorisation and manipulation of algebraic fractions, students compartmentalised knowledge. For example, most students knew how to factorise algebraic expressions, but failed to see the resemblance between algebraic expressions and trigonometric expressions (and consequently, how to factorise trigonometric expressions). Although there was a decrease in the number of Newman errors from the simplification of trigonometric expressions to the proving of trigonometric identities, there was an increase at the comprehension hierarchy, which may be attributed to the fact that the students might have struggled with the concept of “proof”. Additionally, students in this study struggled with the concept of “simpler”. Some students thought that the solution to a simplification question should be more complex than the original question. Nonetheless, with both the simplification of trigonometric expressions and the proving of trigonometric identities it remained a challenge for the students to apply prior knowledge in a new mathematical context such as trigonometry. The significance of the study's findings is that they suggest that teachers re-evaluate how to instruct known mathematical processes and procedures, so as not to compartmentalise mathematical knowledge. Productive struggles may not always produce correct answers; but given sufficient time and appropriate intervention by their teacher, students can build their own knowledge and become independent thinkers who can apply prior knowledge in new contexts such as the simplification of trigonometric expressions and the proving of trigonometric identities. In future research, productive struggles in the simplification of trigonometric expressions and the proving of trigonometric identities should be explored with a bigger, more diverse group of students, taught by more than one teacher at more than one school. In addition, to investigate the long-term effects of productive struggles a study lasting more than six months could be carried out.
Open Access
Introduction to Complex Numbers
(Jonathan Shock, 2020-04) Mokhithi, Mashudu; Shock, Jonathan; Shock, Jonathan
This is an introduction to the mathematics of complex numbers, starting from the very basics of their definitions, up to proving theorems for polynomials. The text covers everything required of most first-year mathematics courses on complex numbers with proofs, where appropriate.
Open Access
Learning to Coordinate Efficiently through Multiagent Soft Q-Learning in the presence of Game-Theoretic Pathologies
(2022) Danisa, Siphelele; Shock, Jonathan; Pretorius, Arnu
In this work we investigate the convergence of multiagent soft Q-learning in continuous games where learning is most likely to be affected by relative overgeneralisation. While this will occur more often in multiagent independent learner problems, it is present in joint-learner problems when information is not used efficiently in the learning process. We first investigate the effect of different samplers and modern strategies of training and evaluating energy-based models on learning to get a sense of whether the pitfall is due to sampling inefficiencies or underlying assumptions of the multiagent soft Q-learning extension (MASQL). We use the word sampler to refer to mechanisms that allow one to get samples from a given (target) distribution. After having understood this pitfall better, we develop opponent modelling approaches with mutual information regularisation. We find that while the former (the use of efficient samplers) is not as helpful as one would wish, the latter (opponent modelling with mutual information regularisation) offers new insights into the required mechanism to solve our problem. The domain in which we work is called the Max of Two Quadratics differential game where two agents need to coordinate in a non-convex landscape, and where learning is impacted by the mentioned pathology, relative overgeneralisation. We close this research investigation by offering a principled prescription on how to best extend single-agent energy-based approaches to multiple agents, which is a novel direction.
Open Access
MAM1000W Calculus Notes
(2013-10) Shock, Jonathan
This resource was developed to be used as self study notes for first-year university mathematics students. The examples and questions contained in this resource ensure gradual growth in depth and understanding of mathematical concepts, and cover a wide range of topics spanning from Integration to the Binomial theorem & Vectors. The notes include graphs & simulated figures that make it easy to visualise the concepts. The notes also have examples and solutions that demonstrate the application of topics. Mathemafrica is a blogging platform for bloggers writing about mathematics within Africa. A multilingual, multiblogger platform, Mathemafrica aims to give a voice to anyone who wants to discuss mathematics and be a source of information and inspiration for anyone who would like to know how mathematics might be relevant to them. The website contains many more examples of introductory calculus, including materials translated into other African languages. Please click on the link in the source field to visit the website.
Open Access
Modeling population dynamics of rhino-poacher interaction across South Africa and the Kruger National Park using ordinary differential equations
(2020) Makic, Vladimir; Shock, Jonathan
In this thesis, a system of ordinary differential equations (ODES) is presented to model the population dynamics between poachers and rhino as a predator-prey system in both South Africa (SA) and the Kruger National Park (KNP). The data used in this thesis consists mainly of government and police reports, as well publications from several NGOs and the limitations caused by this lack of applicable data are explored. The system dynamics are based on Lotka-Volterra differential equations, which are extended to include both a carrying capacity and the Allee effect. This thesis parameterises a model of the dynamics of the interaction between rhino and poachers for some time t and makes predictions based on the interpolation of the available data. The unknown rates and parameters relating to the behaviour of populations R and P are optimised by initially using a combination of educated guesses made from the available data or trial and error until set values are obtained. The remaining unknowns are numerically optimised based on the fixed value parameters. This is considered a constrained system, and the results obtained can only be viewed as constrained predictions based on parameter values obtained by a combination of trial and error and numerical optimisation; namely root mean square (RMS) error considering the available data and model solution at time t. Those parameter values obtained through RMS are regarded as error-minimising parameters within the scope of this research, and make up the final models which are referred to as the models which have been fitted to data. This thesis is an introductory, exploratory work into future attempts at modeling population dynamics with very little or no available data. The models are solved for in a constrained system, limiting the resulting predictions to constrained estimates based on the assigned values to unknown parameters. These solutions predict rhino stabilisation for both models, with active poachers dying out in the KNP but general co-existence observed across SA, within the constrained system.
Open Access
Optimizing COVID-19 control measures using multi-objective deep reinforcement learning
(2023) Folarin, Arinze Lawrence; Shock, Jonathan
A crucial area of global research is the hunt for efficient non-pharmaceutical methods to stop the spread of diseases. Recent research has shown that reinforcement learning can be a helpful tool in the medical industry to ad- dress challenging and delicate issues. The goal of this study is to improve COVID-19 control measures through the use of multi-objective deep re- inforcement learning techniques. The results of two case studies, one using a Pareto conditioned network on COVID-19 data from Belgium and the other using a Deep Q-Network, Goal-DQN, and Non-dominated Sorting Genetic Algorithm (NSGA-II) on COVID-19 data from France, are evaluated using both binomial (Stochastic) and Ordinary Differen- tial Equation mathematical models. The study highlights the potential of multi-objective deep reinforcement learning as a method of optimizing public health interventions by shedding light on the optimum COVID-19 control methods for various scenarios and models. Findings show that the suggested strategies are efficient in figuring out the best preventive actions by striking a balance between two crucial choice difficulties encountered when trying to stop the spread of Covid-19 in particular areas. This study makes a substantial contribution to the ongoing fight against pandemics like the Covid-19 event.
Open Access
Quantum states on spheres in the presence of magnetic fields
(2019) Slayen, Ruach Pillay; Murugan, Jeffrey; Shock, Jonathan
The study of quantum states on the surface of various two-dimensional geometries in the presence of strong magnetic fields has proven vital to the theoretical understanding of the quantum Hall effect. In particular, Haldane’s seminal study of quantum states on the surface of a compact geometry, the sphere, in the presence of a monopole magnetic field, was key to developing an early understanding of the fractional quantum Hall effect. Most of the numerous studies undertaken of similar systems since then have been limited to cases in which the magnetic fields are everywhere constant and perpendicular to the surface on which the charged particles are confined. In this thesis, we study two novel variations of Haldane’s spherical monopole system: the 'squashed sphere’ in the presence of a monopole-like magnetic field, and the sphere in the presence of a dipole magnetic field. In both cases the magnetic field is neither perpendicular nor constant with respect to the surface on which the charged particles are confined. Furthermore, the spherical dipole system has vanishing net magnetic flux. For the 'squashed sphere’ system we find the lowest Landau level single-particle Hilbert space, and it is shown that the effect of the squashing is to localise the particles around the equator. For the spherical dipole system we find the entire single-particle Hilbert space and energy spectrum. We show that in the strong-field limit the spectrum exhibits a Landau level structure, as in the spherical monopole case. Unlike in the spherical monopole case, each Landau level is shown to be infinitely degenerate. The emergence of this Landau level structure is explained by the tendency of a strong dipole field to localise particles at the poles of the sphere.
Open Access
Reinforcement learning algorithms to model learning and decision-making in individuals with depressive disorders
(2022) Niit, Lizelle; Shock, Jonathan
Mental illness causes enormous suffering for many people. Current treatments do not reliably alleviate that suffering. Unclear conceptualisations of mental disorders combined with little knowledge about their aetiology are roadblocks to developing better treatments. This dissertation reviews attempts to use reinforcement learning models to improve the way we conceptualise some of the processes happening in the brain in mental illness. The hope is that more clearly defining the problems we are dealing with will eventually have a positive impact on our ability to diagnose and treat them. I start by giving an overview of the reinforcement learning framework, and detail some of the reinforcement learning models that have been used to understand mental illness better. I explain the statistical techniques used to compare these models and to estimate parameters once a model has been chosen. This leads in to a survey of what researchers have learned about human behaviour using these techniques. I focus particularly on results related to depression. I argue that key parameters like learning rate and reward sensitivity are closely linked to depressive symptoms. Finally, I speculate about the impact that knowledge of this kind may have on the development of better diagnosis and treatment for mental illness in general and depression specifically.
Open Access
Saliency Mapping in Convolutional Neural Networks to Determine Brain Age Trajectories
(2022) Taylor, Daniel; Shock, Jonathan; Moodley, Deshendran
Brain Age (BA) estimation via Deep Learning has become a strong and reliable bio-marker for brain health, but the black-box nature of Neural Networks does not easily allow insight into the causal features of brain ageing. In this work, a ResNet model was trained as a BA regressor on T1 structural brain MRI volumes from a small cross-sectional cohort of 524 individuals. Using Layer-wise Relevance Propagation (LRP) and DeepLIFT saliency mapping techniques, analyses were performed on the trained model to determine the most revealing structures over the course of brain ageing for the network, and compare these between the saliency mapping techniques. This work shows the change in attribution of relevance to di erent brain regions through the course of ageing. A tripartite pattern of relevance attribution to brain regions emerges. Some regions increase in relevance with age (e.g. the right Transverse Temporal Gyrus, known to be a ected by healthy ageing); some decrease in relevance with age (e.g. the right Fourth Ventricle, known to dilate with age); and others remained consistently relevant across ages. This work also examines the e ect of Brain Age Delta (DBA) on the distribution of relevance within the brain volume, for both older and younger individuals. It is hoped that these ndings will provide clinically relevant region-wise trajectories for normal brain ageing, and a baseline against which to compare brain ageing trajectories.

Browsing by Author "Shock, Jonathan"

Results Per Page

Sort Options