Meta Learning for Reinforcement Learning

Brief description of the context of the research including potential impact

Meta learning, or 'learning to learn', offers potential for us to move away from hand-crafted algorithms in machine learning. Based on the fact that learned algorithms thus far have proven more effective than hand-crafted features in practically all scenarios, it is only reasonable to expect that this holds in terms of a machine's learning process as well. This has the potential to drastically improve our machine learning models in practically all settings and possible enable us to work with ML algorithms in currently non-practical settings.

Aims and Objectives

The primary aim of our work is to learn effective algorithms for solving reinforcement learning tasks, though it is hoped that at least some of the work throughout this DPhil will be generalisable to other fields of machine learning too. In particular, my current project is focused on attempting to learn a more effective optimiser to replace handcrafted update rules that will, hopefully, generalise within reinforcement learning.

Novelty of the research methodology

There is a range of novelty on offer in this current work; we are exploring different methods for meta learning than the current SOTA (employing evolutionary strategies) and exploring the different effects of conditioning features on performance. This is also leading to some interesting analysis on the impact of a meta learned optimiser on, eg, plasticity loss in neural networks, particularly in reinforcement learning.

Alignment to EPSRC's strategies and research areas (which EPSRC research area the project relates to) Further information on the areas can be found on http://www.epsrc.ac.uk/research/ourportfolio/researchareas/ http://www.epsrc.ac.uk/research/ourportfolio/researchareas/)

This is closely aligned to pushing AI technologies, which is one of EPSRC's research areas.

Any companies or collaborators involved

Currently, Pablo Castro (Deepmind) is working with...