Ever since the Deepmine Alpha Go paper came out, I have always been fascinated by what reinforcement learning could do. In my opinion, it’s the closest branch of AI to Artificial General Intelligence. Because no matter what your goal is, as long as you determined the reward correctly, it will eventually help you to find the “optimal” solution. What it lacks, perhaps, is that it takes a long time to learn. For example, human babies don’t have to stumble millions of times to learn how to walk. And it’s even more evident for a deer, where they pretty much know how to walk after they were born. Now, that is not too say evolution didn’t give some prebuilt model for the deer to use, but maybe we can make some pretrained model for AI to know how to walk, like the modules they insert into Neo to give him fighting abilities.

When I set out to learn reinforcement learning, there wasn’t much formalized material. There is Sutton and Barto’s book, but I found it was and still written for experts. The exercises they give are necessary for me to understand what’s going on, but there is no explanation given. There is too much “for obvious reasons” for me to understand some basic concepts, like how do I calculate state value for each grid. There is a formula for it, but no detailed walkthrough to how to assign those values. Then there are David Silver’s lectures, which shined more light for me than the book, but there is also too much detail left unexplained. So, I have searched the internet for some tutorials and finally landed on this medium article. https://medium.com/free-code-camp/an-introduction-to-reinforcement-learning-4339519de419. It has enough little details for me to work out how to get to individual values, starting from what’s a Q table and how implement it in python. I highly recommend going through the tutorial and exercises on Google Colab and make sure you understand how the environment is used in the Gym package.

Anyway, I’m still working through all of the code and understanding it. I also recommend a Lego Mindstorm set to give you a physical manifestation to work with. It can help you familiarize with how to apply it in the physical world. The basic robotics with a stable, not bipedal, robot, also makes learning easier. Make sure you use a customize python package like ev3dev so you can control the robot with python scripts. https://www.ev3dev.org/ Now that’s how you have endless, time sucking fun!