Chapter 2, Getting Started with the Q-Learning Algorithm

1. Generally speaking, a control process is designed to optimize a value or a set of values within a set of limitations.
2. A Markov chain does not incorporate actions or rewards; it only has states and events that will lead from one state to the next.
3. The Markov property is the certainty that knowledge of a system's future states does not depend on knowledge of past states, but only on the current state.
4. The Taxi-v2 environment has 500 states based on the values the state variables can take. State variables are the location of the taxi, the location of the destination, and the location of the passenger.
5. We include these states for simplicity in enumerating the state space. They are unreachable in the task because, in some cases, the environment reaches a terminal state before they can be reached. For example, when the taxi...