Learning Reinforcement Learning

I've been learning reinforcement learning (RL) recently. I'm starting this blog to share some of the useful resources I've found and to show some of the work I do. I'm very new to RL and by no means an authority on the field. This first post will focus on what on the learning RL resources I've found helpful.

A Quick Introduction

• Andrej Karpathy's post on  learning Pong. Karpathy is an excellent teacher. If you're interested in convolutional neural nets, image recognition, and better understanding gradients his lectures from the Stanford class he used to teach are also great.
• Demystifying Deep Learning. Post on the basics of RL with a focus on DeepMind's Deep Q network which performed well on a bunch of Atari games in 2015. Explains some of the basics in a quick, yet informative way.
• AlphaGo dominating chess after 4 hours of training. News article about another success for AlphaGo. RL is a tricky field. Not as much impact in the real world compared to other machine learning disciplines. Much of the results of RL are limited to very specific problems. It can be a bit discouraging at times but then DeepMind does something crazy like learns Go from scratch or crushes chess.

A First Course

• David Silver (of DeepMind fame) has a great series of lectures from his intro to RL course. He recommends Sutton and Barto's Reinforcement Learning: An Introduction as an accompanying text book. The lecture and the textbook are a great place to get started.
• RL Study Group on Discord: Has a recommended path for learning RL, discussion, recommended papers, and more. The first 7 or so tasks start with much of what Silver covers and has some recommended exercises.
• Denny Britz's RL Github. Some exercises to get some experience with the basics of RL and see how they are implemented in Python and TensorFlow
• OpenAI's gym environment. Lot of neat stuff here to implement RL algorithms in some easy to install and code examples. For most of the games here you can try your own approach or search online to see how others have tackled some of the problems.

Going Further

I'm at this stage now so don't have much of a recommendation at this time. If anyone has some thoughts, feel free to comment. Some of the things I'm trying to do:

• Run my own experiments. I'll have some posts on this soon.
• Watching videos from U Cal's Deep Reinforcement Learning course
• Checking out what people recommend on reddit
• Reading recommended papers
• Reading blogs
• Keeping up with the new tasks on the RL study group.