Hands-On Intelligent Agents with OpenAI Gym

▶Book Description
Many real-world problems can be broken down into tasks that require a series of decisions to be made or actions to be taken. The ability to solve such tasks without a machine being programmed requires a machine to be artificially intelligent and capable of learning to adapt. This book is an easy-to-follow guide to implementing learning algorithms for machine software agents in order to solve discrete or continuous sequential decision making and control tasks.

Hands-On Intelligent Agents with OpenAI Gym takes you through the process of building intelligent agent algorithms using deep reinforcement learning starting from the implementation of the building blocks for configuring, training, logging, visualizing, testing, and monitoring the agent. You will walk through the process of building intelligent agents from scratch to perform a variety of tasks. In the closing chapters, the book provides an overview of the latest learning environments and learning algorithms, along with pointers to more resources that will help you take your deep reinforcement learning skills to the next level.

▶What You Will Learn
⦁ Explore intelligent agents and learning environments
⦁ Understand the basics of RL and deep RL
⦁ Get started with OpenAI Gym and PyTorch for deep reinforcement learning
⦁ Discover deep Q learning agents to solve discrete optimal control tasks
⦁ Create custom learning environments for real-world problems
⦁ Apply a deep actor-critic agent to drive a car autonomously in CARLA
⦁ Use the latest learning environments and algorithms to upgrade your intelligent agent development skills

▶Key Features
⦁ Explore the OpenAI Gym toolkit and interface to use over 700 learning tasks
⦁ Implement agents to solve simple to complex AI problems
⦁ Study learning environments and discover how to create your own

▶Who This Book Is For
If you’re a student, game/machine learning developer, or AI enthusiast looking to get started with building intelligent agents and algorithms to solve a variety of problems with the OpenAI Gym interface, this book is for you. You will also find this book useful if you want to learn how to build deep reinforcement learning-based agents to solve problems in your domain of interest. Though the book covers all the basic concepts that you need to know, some working knowledge of Python programming language will help you get the most out of it.

▶What this book covers
⦁ Chapter 1, Introduction to Intelligent Agents and Learning Environments, which enables the development of several AI systems. It sheds light on the important features of the toolkit, which provides you with endless opportunities to create autonomous intelligent agents to solve several algorithmic tasks, games, and control tasks. By the end of this chapter, you will know enough to create an instance of a Gym environment using Python yourself.

⦁ Chapter 2, Reinforcement Learning and Deep Reinforcement Learning, provides a concise explanation of the basic terminologies and concepts in reinforcement learning. The chapter will give you a good understanding of the basic reinforcement learning framework for developing AI agents. The chapter will also introduce deep reinforcement learning and provide you with a flavor of the types of advanced problem the algorithms enable you to solve.

⦁ Chapter 3, Getting Started with OpenAI Gym and Deep Reinforcement Learning, jumps right in and gets your development machine/computer ready with all the required installations and configurations needed for using the learning environments as well as PyTorch for developing deep learning algorithms.

⦁ Chapter 4, Exploring the Gym and its Features, walks you through the inventory of learning environments available with the Gym library starting with the overview of how the environments are classified and named which will help you choose the correct version and type of environments from the 700+ learning environments available. You will then learn to explore the Gym, test out any of the environment you would like to, understand the interface and description of various environments.

⦁ Chapter 5, Implementing your First Learning Agent –. Solving the Mountain Car problem, explains how to implement an AI agent using reinforcement learning to solve the mountain car problem. You will implement the agent, train it, and see it improve on its own. The implementation details will enable you to apply the concepts to develop and train an agent to solve various other tasks and/or games.

⦁ Chapter 6, Implementing an Intelligent Agent for Optimal Control using Deep Q-Learning, covers various methods to improve Q-learning including action-value function approximation using deep neural network, experience replay, target networks and also the necessary utilities and building-blocks that are useful for training and testing deep reinforcement learning agents in general. You will implement a DQN based intelligent agent for taking optimal discrete control actions and train it to play several Atari games and watch the agent's performance.

⦁ Chapter 7, Creating Custom OpenAI Gym Environments –. Carla Driving Simulator, will teach you how to convert a real-world problem into a learning environment with interfaces compatible with the OpenAI Gym. You will learn the anatomy of Gym environments and create your custom learning environment based on the Carla simulator that can be registered with the Gym and used for training agents that we develop.

⦁ Chapter 8, Implementing an Intelligent & Autonomous Car Driving Agent using Deep Actor-Critic Algorithm, teaches you the fundamentals of the Policy Gradient based reinforcement learning algorithms and helps you intuitively understand the deep n-step advantage actorcritic algorithm. You will then learn to implement a super-intelligent agent that can drive a car autonomously in the Carla simulator using both the synchronous as well as asynchronous implementation of the deep n-step advantage actor-critic algorithm.

⦁ Chapter 9, Exploring the Learning Environment Landscape –. Roboschool, Gym-Retro, StarCraft-II, DeepMindLab, takes you beyond the Gym and shows you around other well developed suite of learning environments that you can use to train your intelligent agents. You will understand and learn to use the various Roboschool environments, the Gym Retro environments, the very popular Star Craft II environment and the DeepMind Lab environments.

⦁ Chapter 10, Exploring the Learning Algorithm Landscape –. DDPG (Actor-Critic), PPO (Policy- Gradient), Rainbow (Value-Based), Provides insights into latest deep reinforcement learning algorithms with their fundamentals demystified based on what you learned in the previous chapters of this book. You will get a quick understanding of the core concepts behind the best algorithms in the three different classes of deep reinforcement learning algorithms namely: The actor-critic based Deep Deterministic Policy Gradient (DDPG) algorithm, the Policy Gradient based Proximal Policy Optimization (PPO) and the value based Rainbow algorithm.