Research talk: Breaking the deadly triad with a target network
- Shangtong Zhang | Oxford University
- Microsoft Research Summit 2021 | Reinforcement Learning
The deadly triad refers to the instability of an off-policy reinforcement learning (RL) algorithm when it employs function approximation and bootstrapping simultaneously, and this is a major challenge in off-policy RL. Join PhD student Shangtong Zhang, from the WhiRL group at the University of Oxford, to learn how the target network can be used as a tool for theoretically breaking the deadly triad. Together, you’ll explore how to theoretically understand the conventional wisdom that a target network stabilizes training, a novel target network update rule that augments the commonly used Polyak-averaging style update with two projections, and how a target network can be used in linear off-policy RL algorithms, in both prediction and control settings, as well as both discounted and average-reward Markov decision processes.
Learn more about the 2021 Microsoft Research Summit: https://Aka.ms/researchsummit (opens in new tab)
-
-
Shangtong Zhang
PhD Student
Oxford University
-
-
Reinforcement Learning
-
Opening remarks: Reinforcement Learning
- Katja Hofmann
-
-
-
-
Research talk: Evaluating human-like navigation in 3D video games
- Raluca Georgescu,
- Ida Momennejad
-
Research talk: Maia Chess: A human-like neural network chess engine
- Reid McIlroy-Young
-
Fireside chat: Opportunities and challenges in human-oriented AI
- Ashley Llorens,
- Katja Hofmann,
- Siddhartha Sen
-
Research talk: Making deep reinforcement learning industrially applicable
- Jiang Bian,
- Tie-Yan Liu
-
Panel: Generalization in reinforcement learning
- Mingfei Sun,
- Roberta Raileanu,
- Wendelin Böhmer
-
Research talk: Project Dexter: Machine learning and automatic decision-making for robotic manipulation
- Andrey Kolobov,
- Ching-An Cheng
-
-
-
Research talk: Breaking the deadly triad with a target network
- Shangtong Zhang
-
Panel: The future of reinforcement learning
- Geoff Gordon,
- Emma Brunskill,
- Craig Boutilier
-
Closing remarks: Reinforcement Learning
- John Langford
