Notes and further reading

1. The extra complication is that we can have more than one limiting distribution; that is, there is more than one solution to Eq. 12. Which limiting distribution we end up in after iterating for an infinite amount of time will then depend upon the details of the starting distribution .
2. Very high-order discrete Markov models have been used to model language generation. This approach to modeling language also has some similarities with how large language models (LLMs) work in that LLMs also use a large number of the preceding states to predict the next word. However, the inclusion of the self-attention mechanism in LLMs makes them significantly different and superior to high-order Markov models for language generation.
3. One of the earliest approaches to doing this was by A.E. Raftery, A Model for High-Order Markov Chains, Journal of the Royal Statistical Society, Series B, 47(3):528-539 (1985).
4. I learned about HMMs in the context of DNA sequence...