To better understand how to overcome the problems that game developers are currently facing, we need to dig a little bit into the history of video game development and take a look at the problems and their solutions that were so important at the time. Some of them were so avant-garde that they actually changed the entire history of video game design itself, and we still use the same methods today to create unique and enjoyable games.

One of the first relevant marks that is always worth mentioning when talking about game AI is computer chess programmed to compete against humans. It was the perfect game to start experimenting with artificial intelligence, because chess usually requires a lot of thought and planning ahead, something that a computer couldn't do at the time because it was necessary to have human features in order to successfully play and win the game. So, the first step was to make it able for the computer to process the game rules and think for itself in order to make a good judgement of the next move that the computer should do to achieve the final goal, that is, winning by checkmating. The problem is that chess has many possibilities; so, even if the computer had a perfect strategy to beat the game, it was necessary to recalculate that strategy, adapting it, changing, or even creating a new one every time something went wrong with the first strategy.

Humans can play differently every time; this makes it a huge task for the programmers to input all the possible data into the computer in order to win the game. So, writing all the possibilities that could exist wasn't a viable solution, and because of that, the programmers needed to think again about the problem. Then, one day, they finally came out with a better solution, that is, to make the computer decide for itself every turn, choosing the most plausible option for each turn; that way, the computer could adapt to any possibility in the game. Yet, this involved another problem–the computer would only think the short-term moves, and not create any plans to defeat the human in the future moves; so, it was easy to play against it, but at least we started to have something going on. It was decades later that someone defined the word Artificial Intelligence (AI) by solving the first problem that many researchers had by trying to create a computer that was capable of defeating a human player. Arthur Samuel is the person responsible for creating a computer that could learn for itself and memorize all the possible combinations. That way, there wasn't necessarily any human intervention and the computer could actually think on its own, and that was a huge step that is still impressive even by today's standards.

Now, let's move to the video game industry and analyze how the first enemies and game obstacles were programmed; was it that different from what we are doing now? Let's find out.

Single-player games with AI enemies started to appear in the 1970s, and soon, some games started to elevate the quality and expectations of what defines a video game AI. Some of those examples were released for arcade machines, such as Speed Race from Taito (a racing video game), or Qwak (a duck hunting game using a light gun), and Pursuit (an aircraft fighter) both from Atari. Other notable examples are the text-based games released for the first personal computers, such as Hunt the Wumpus and Star Trek, which also had enemies. What made those games so enjoyable was precisely that the AI enemies that didn't reacted like any other before because they had random elements mixed with the traditional stored patterns, making them unpredictable, and hence providing a unique experience every time you played the game. However, that was only possible due to the incorporation of microprocessors that expanded the capabilities of a programmer at that time. Space Invaders brought the movement patterns and Galaxian improved and added more variety, making the AI even more complex. PAC-MAN later on brought movement patterns to the maze genre.

The influence that the AI design in PAC-MAN had is just as significant as the influence of the game itself. This classic arcade game makes the player believe that the enemies in the game are chasing him, but not in a crude manner. The ghosts are chasing the player (or evading the player) in a different way as if they have an individual personality. This gives people the illusion that they are actually playing against four or five individual ghosts rather than copies of the same computer enemy.

After that, Karate Champ introduced the first AI fighting character and Dragon Quest introduced the tactical system for the RPG genre; over the years, the list of games that explored artificial intelligence and used it to create unique game concepts kept expanding, and all of that came from a single question, how can we make a computer capable of beating a human in a game?

All the games mentioned above are of a different genre, and they are unique in their style, but all of them used the same method for the AI called finite-state machine (FSM). Here, the programmer inputs all the behaviors necessary for the computer to challenge the player, just like the computer that first played chess. The programmer defined exactly how the computer should behave on different occasions in order to move, avoid, attack, or perform any other behavior to challenge the player, and that method is used even in the latest big budget games of

Programmers face many challenges while developing an AI character, but one of the greatest challenges is adapting the AI movement and behavior in relation to what the player is currently doing, or will do in future actions. The difficulty exists because the AI is programmed with predetermined states, using probability or possibility maps in order to adapt their movement and behavior according to the player. This technique can become very complex if the programmer extends the possibilities of the AI decisions, just like the chess machine that has all the possible situations that may occur in the game.

It's a huge task for the programmer because it's necessary to determine what the player can do and how the AI will react to each action of the player, and that takes a lot of CPU power. To overcome that challenge, programmers started to mix possibility maps with probabilities and perform other techniques that let the AI decide for itself on how it should react according to the player's actions. These factors are important to be considered while developing an AI that elevates the game quality as we are about to discover.

Games kept evolving and players got even more exigent, not only with the visual quality but also with the capabilities of the AI enemies and the allied characters. To deliver new games that took into consideration the player expectations, programmers started to write even more states for each character, creating new possibilities and more engaging enemies, implementing important allied characters, which meant more things for the player to do, and creating a lot more features that helped redefine different genres and created new ones. Of course, this was also possible because technology kept improving, allowing developers to explore even more artificial intelligence in video games. A great example of this that is worth mentioning is Metal Gear Solid, the game that brought a new genre to the video game industry by implementing stealth elements, instead of the popular straightforward shooting. However, those elements couldn't be fully explored as Hideo Kojima intended because of the hardware limitations at the time. Jumping forward from the third to the fifth generation of consoles, Konami and Hideo Kojima presented the same title, but this time with a lot more interactions, possibilities, and behaviors from the AI elements of the game, making it so successful and important in video game history that it's easy to see its influence in a large number of games that came after Metal Gear Solid:

The game in the preceding screenshot implemented visual and audio awareness for the enemy AI, a feature that later on we'll explore in detail in this book. This feature established the genre that we know today as a stealth game. So, the game uses Path Finding and a FSM, features that were already known from the beginning of the video game industry; But, in order to create something new, they also created new features, such as interaction with the environment, navigation behavior, visual/audio awareness, and AI interaction; a lot of things that didn't existed at the time but that are widely used today in different game genres, such as sports, racing, fighting, or FPS games, were also introduced:

After that huge step for game design, developers still faced other problems, or should I say, these new possibilities brought even more problems, because they were not perfect. The AI still didn't react as a real person, and many other elements was necessary to be implemented, not only in stealth games, but in all other genres, and one in particular-needed to improve their AI to make the game feel realistic.

We are talking about sports games, especially those that tried to simulate real-world team behaviors, such as basketball or football. interaction with the player is not the only thing that we need to care about; we left chess long time ago, where it was 1 versus 1. Now, we want more, and watching other games get realistic AI behaviors, sport fanatics started to ask for the same features in their

favorite games; after all, those games was based on real-world events, and for that reason, the AI should react as realistically as possible.

At this point, developers and game designers started to take into consideration AI interaction with itself, and that just like the enemies from PAC-MAN, the player should get the impression that each character in the game thinks for itself and reacts differently to the others. If we analyze it closely, the AI that is present in a sports game is structured like an FPS or RTS game, using different animation states, general movements, interactions, individual decisions, and finally tactics and collective decisions. So, it shouldn't be a surprise that sports games could reach the same level of realism as the other genres that had already greatly evolved in terms of AI development. However, there are few problems that only sports games had at the time: how to make so many characters on the same screen react differently but at the same time work together to achieve the same objective. With this problem in mind, developers started to improve the individual behaviors of each character, not only for the AI that was playing against the player but also for the AI that was playing alongside the player. Once again, Finite State Machines made up a crucial part of Artificial Intelligence, but the special touch that helped to create a realistic approach in the sports genre was the anticipation and awareness used in stealth games. The computer needed to calculate what the player was doing, where the ball was going, and coordinate all of that, as well as give a false impression of a team mindset toward the same plan. Combining the new features used in the new genre of stealth games with a vast number of characters on the same screen, it was possible to innovate the sports genre by creating a sports simulation type of game, which has gained so much popularity over the years. This helps us to understand that we can use almost the same methods for any type of game, even if it looks completely different; the core principles that we saw in the computer that played chess is still valuable to the sports game released 30 years later.

Let's move on to our last example, which also has great value in terms of how an AI character should behave to make it more realistic: the game is F.E.A.R., developed by Monolith Productions. What made this game so special in terms of Artificial Intelligence was the dialog between the enemy characters. While it wasn't an improvement from a technical point of view, it was definitely something that helped to showcase all of the development work that was put into the characters' AI, and this is so crucial because if the AI doesn't say it, it didn't happen. This is an important factor to take into consideration while creating a realistic AI character, giving the illusion that it's real; that means the false impression that the computer reacts like humans, and humans interact, so the AI should do the same. Not only did the dialog help to create a human-like atmosphere, it also helped to exhale all of the development put on the character that otherwise the player wouldn't notice was there. When the AI detects the player for the first time, it shouts that it found the player; when the AI loses sight of the player, it expresses that. When the squad of AI's are trying to find the player or ambush him, they speak about that, leaving the player imagining that the enemy is really capable of thinking and planning against him. Why is this so important? Because if we only had numbers and mathematical equations for the characters, they will react that way, without any human features, just math, and to make it look more human, it's necessary to input mistakes, errors, and dialog into the character AI, just to distract the player from the fact that he's playing against a machine.

The history of video game artificial intelligence is still far from perfect, and it's possible that it will take us decades to improve just a little bit more from what we achieved between the early 1950s and this present day, so don't be afraid of exploring what you are about to learn, combine, change, or delete some of the things to find different results, because great games did it in the past and they had a lot of success with it.

In this chapter, you learned about the impact of AI on the video game history, how everything started from a simple idea to have a computer to compete against humans in traditional games, and how that naturally evolved into the world of video games. You also learned about the challenges and difficulties that were present since the day one, and how coincidentally, programmers kept facing and still face the same problems. In the next chapter, we'll start from that precise point, the most used technique and the technique that caused a lot of debate and evolution that was present on past games as well on present and futures ones.