The purposes of this chapter are manifold. We will introduce many topics that are essential to the understanding of reinforcement problems and the first algorithms that are used to solve them. Whereas, in the previous chapters, we talked about reinforcement learning (RL) from a broad and non-technical point of view, here, we will formalize this understanding to develop the first algorithms to solve a simple game.
The RL problem can be formulated as a Markov decision process (MDP), a framework that provides a formalization of the key elements of RL, such as value functions and the expected reward. RL algorithms can then be created using these mathematical components. They differ from each other by how these components are combined and on the assumptions made while designing them.
For this reason, as we'll see in this chapter, RL algorithms...
