▶Book Description
Recent developments in reinforcement learning (RL), combined with deep learning (DL), have seen unprecedented progress made towards training agents to solve complex problems in a human-like way. Google's use of algorithms to play and defeat the well-known Atari arcade games has propelled the field to prominence, and researchers are generating new ideas at a rapid pace.
Deep Reinforcement Learning Hands-On is a comprehensive guide to the very latest DL tools and their limitations. You will evaluate methods including Cross-entropy and policy gradients, before applying them to real-world environments. Take on both the Atari set of virtual games and family favorites such as Connect4. The book provides an introduction to the basics of RL, giving you the know-how to code intelligent learning agents to take on a formidable array of practical tasks. Discover how to implement Q-learning on 'grid world' environments, teach your agent to buy and trade stocks, and find out how natural language models are driving the boom in chatbots.
▶What You Will Learn
⦁ Understand the DL context of RL and implement complex DL models
⦁ Learn the foundation of RL: Markov decision processes
⦁ Evaluate RL methods including Cross-entropy, DQN, Actor-Critic, TRPO, PPO, DDPG, D4PG and others
⦁ Discover how to deal with discrete and continuous action spaces in various environments
⦁ Defeat Atari arcade games using the value iteration method
⦁ Create your own OpenAI Gym environment to train a stock trading agent
⦁ Teach your agent to play Connect4 using AlphaGo Zero
⦁ Explore the very latest deep RL research on topics including AI-driven chatbots
▶Key Features
⦁ Explore deep reinforcement learning (RL), from the first principles to the latest algorithms
⦁ Evaluate high-profile RL methods, including value iteration, deep Q-networks, policy gradients, TRPO, PPO, DDPG, D4PG, evolution strategies and genetic algorithms
⦁ Keep up with the very latest industry developments, including AI-driven chatbots
▶Who This Book Is For
The main target audience are people who have some knowledge in Machine Learning, but interested to get a practical understanding of the Reinforcement Learning domain. A reader should be familiar with Python and the basics of deep learning and machine learning. Understanding of statistics and probability will be a plus, but is not absolutely essential for understanding most of the book's material.
▶What this book covers
⦁ Chapter 1, What is Reinforcement Learning?, contains introduction to RL ideas and main formal models.
⦁ Chapter 2, OpenAI Gym, introduces the reader to the practical aspect of RL, using open-source library gym.
⦁ Chapter 3, Deep Learning with PyTorch, gives a quick overview of the PyTorch library.
⦁ Chapter 4, The Cross-Entropy Method, introduces you to one of the simplest methods of RL to give you the feeling of RL methods and problems.
⦁ Chapter 5, Tabular Learning and the Bellman Equation, gives an introduction to the Value-based family of RL methods.
⦁ Chapter 6, Deep Q-Networks, describes DQN, the extension of basic Value-based methods, allowing to solve complicated environment.
⦁ Chapter 7, DQN Extensions, gives a detailed overview of modern extension to the DQN method, to improve its stability and convergence in complex environments.
⦁ Chapter 8, Stocks Trading Using RL, is the first practical project, applying the DQN method to stock trading.
⦁ Chapter 9, Policy Gradients –- An Alternative, introduces another family of RL methods, based on policy learning.
⦁ Chapter 10, The Actor-Critic Method, describes one of the most widely used method in RL.
⦁ Chapter 11, Asynchronous Advantage Actor-Critic, extends Actor-Critic with parallel environment communication, to improve stability and convergence.
⦁ Chapter 12, Chatbots Training with RL, is the second project, showing how to apply RL methods to NLP problems.
⦁ Chapter 13, Web Navigation, is another long project, applying RL to web page navigation, using MiniWoB set of tasks.
⦁ Chapter 14, Continuous Action Space, describes the specifics of environments, using continuous action spaces and various methods.
⦁ Chapter 15, Trust Regions –- TRPO, PPO, and ACKTR, is yet another chapter about continuous action spaces describing "Trust region" set of methods.
⦁ Chapter 16, Black-Box Optimization in RL, shows another set of methods that don't use gradients in explicit form.
⦁ Chapter 17, Beyond Model-Free –- Imagination, introduces model-based approach to RL, using recent research results about imagination in RL.
⦁ Chapter 18, AlphaGo Zero, describes the AlphaGo Zero method applied to game Connect Four.
