Updating the Bellman equation iteratively

You may recall that a random approach to solving the taxi cab simulation took our agent about 6,000 time steps. Sometimes, out of sheer luck, you may be able to solve it under 2,000 time steps. However, we can further tip the odds in our favor by implementing a version of the Bellman equation. This approach will essentially allow our agent to remember its actions and corresponding rewards per state by using a Q-table. We can implement this Q-table on Python using a NumPy array, with dimensions corresponding to our observation space (the number of different possible states) and action space (the number of different possible actions our agent can make) in the taxi cab environment. Recall that the taxi cab simulation has an environment space of 500 and an action space of six, making our Q-table a matrix of 500 rows and six columns. We can...