Statistics of character modeling

We often distinguish words and numbers as being in different realms. As it happens, they are not so far apart. Everything can be deconstructed using the universal language of mathematics. This is quite a fortunate property of our reality, not just for the pleasure of modeling statistical distributions over sequences of characters. However, since we are on the topic, we will go ahead and define the concept of language models. In essence, language models follow Bayesian logic that relates the probability of posterior events (or tokens to come) as a function of prior occurrences (tokens that came). With such an assumption, we are able to construct a feature space corresponding to the statistical distribution of words over a period of time. The RNNs we will build shortly will each construct a unique feature space of probability distributions. Then...