The Markov model is a stochastic model in which the state of the random variable at the next instance of time depends only on the outcome of the random variable at the current time. The simplest kind of Markov model is a Markov chain, which we discussed in Chapter 1, Introduction to Markov Process.
Suppose we have a set of sequential observations (x1,. . ., xn) obeying the Markov property, then we can state the joint probability distribution for N observations as the following:
Graphical representation of a first-order Markov chain in which the distribution of the current observation is conditioned on the value of the previous observation
The preceding representation of the Markov chain is different from the representations we saw earlier. In this representation, the observations are presented as nodes and the edges represent conditional probability between...
