Chapter 1, Introduction to Intelligent Agents and Learning Environments, which enables the development of several AI systems. It sheds light on the important features of the toolkit, which provides you with endless opportunities to create autonomous intelligent agents to solve several algorithmic tasks, games, and control tasks. By the end of this chapter, you will know enough to create an instance of a Gym environment using Python yourself.
Chapter 2, Reinforcement Learning and Deep Reinforcement Learning, provides a concise
explanation of the basic terminologies and concepts in reinforcement learning. The chapter
will give you a good understanding of the basic reinforcement learning framework for
developing AI agents. The chapter will also introduce deep reinforcement learning and
provide you with a flavor of the types of advanced problem the algorithms enable you to
solve.
Chapter 3, Getting Started with OpenAI Gym and Deep Reinforcement Learning, jumps right in and gets your development machine/computer ready with all the required installations and configurations needed for using the learning environments as well as PyTorch for developing deep learning algorithms.
Chapter 4, Exploring the Gym and its Features, walks you through the inventory of learning environments available with the Gym library starting with the overview of how the environments are classified and named which will help you choose the correct version and type of environments from the 700+ learning environments available. You will then learn to explore the Gym, test out any of the environment you would like to, understand the interface and description of various environments.
Chapter 5, Implementing your First Learning Agent – Solving the Mountain Car problem, explains how to implement an AI agent using reinforcement learning to solve the mountain
car problem. You will implement the agent, train it, and see it improve on its own. The
implementation details will enable you to apply the concepts to develop and train an agent
to solve various other tasks and/or games.
Chapter 6, Implementing an Intelligent Agent for Optimal Control using Deep Q-Learning, covers various methods to improve Q-learning including action-value function approximation using deep neural network, experience replay, target networks and also the necessary utilities and building-blocks that are useful for training and testing deep reinforcement learning agents in general. You will implement a DQN based intelligent agent for taking optimal discrete control actions and train it to play several Atari games and watch the agent's performance.
Chapter 7, Creating Custom OpenAI Gym Environments – Carla Driving Simulator, will teach you how to convert a real-world problem into a learning environment with interfaces compatible with the OpenAI Gym. You will learn the anatomy of Gym environments and create your custom learning environment based on the Carla simulator that can be registered with the Gym and used for training agents that we develop.
Chapter 8, Implementing an Intelligent & Autonomous Car Driving Agent using Deep Actor-Critic Algorithm, teaches you the fundamentals of the Policy Gradient based reinforcement learning algorithms and helps you intuitively understand the deep n-step advantage actor-critic algorithm. You will then learn to implement a super-intelligent agent that can drive a car autonomously in the Carla simulator using both the synchronous as well as asynchronous implementation of the deep n-step advantage actor-critic algorithm.
Chapter 9, Exploring the Learning Environment Landscape – Roboschool, Gym-Retro, StarCraft-II, DeepMindLab, takes you beyond the Gym and shows you around other well developed suite of learning environments that you can use to train your intelligent agents. You will understand and learn to use the various Roboschool environments, the Gym Retro environments, the very popular Star Craft II environment and the DeepMind Lab environments.
Chapter 10, Exploring the Learning Algorithm Landscape – DDPG (Actor-Critic), PPO (Policy-Gradient), Rainbow (Value-Based), Provides insights into latest deep reinforcement learning algorithms with their fundamentals demystified based on what you learned in the previous chapters of this book. You will get a quick understanding of the core concepts behind the best algorithms in the three different classes of deep reinforcement learning algorithms namely: The actor-critic based Deep Deterministic Policy Gradient (DDPG) algorithm, the Policy Gradient based Proximal Policy Optimization (PPO) and the value based Rainbow algorithm.
