Chapter 5: Simulation-Based Markov Decision Processes

Markov Decision Processes (MDPs) model decision-making in situations where outcomes are partly random and partly under the control of a decision maker. An MDP is a stochastic process characterized by five elements: decision epochs, states, actions, transition probability, and reward. The characteristic elements of a Markovian process are the states in which the system finds itself and the available actions that the decision maker can carry out on those states. These elements identify two sets: the set of states in which the system can be found and the set of actions available for each specific state. The action chosen by the decision maker determines a random response from the system, which brings it into a new state. This transition returns a reward that the decision maker can use to evaluate the goodness of their choice. In this chapter, we will learn how to deal with decision-making processes with Markov chains. We will analyze...