What is an immersive experience?

Immersion is a core concept of how we experience the world around us. It can be minimal (sitting in a park and reading a book) or maximal (going scuba diving in the ocean as you feel the weightlessness from the water pressure pushing against your body), but the fact remains that immersion is a constant in our lives. Quite frankly, we don’t have any concept of what a lack of immersion is because the experiences we have involve some level of immersion.

Medical students are trained to test the functions of the human body so that patients can have a fully immersive experience. As the body declines due to age and disease, we see that things become less immersive, and ultimately, quality of life diminishes. Immersion affects our perceptions and informs our reality to an extent. Nevertheless, there is no box that we can put the concept of immersion into because it is all-encompassing.

The Merriam-Webster Dictionary defines immersion as “a state of being deeply engaged or involved, deep mental involvement.” We can also use the more literal definition: “to plunge into something that surrounds or covers especially: to plunge or dip into a fluid.” Both definitions overlap in many ways because they allude to an ever-present stimulus. When we talk about immersive experiences, we are referring to the concept of how the surrounding environment provides stimuli that inform our perceptions. In those conversations, we often describe what we see, smell, feel, hear, and believe to be true based on what engaged our senses at that moment.

An immersive experience is an illusion that makes you feel like you are inside or part of an environment. We perceive the environment as tangible (real), but it is intangible. This environment engages your senses through the use of technology and feedback to mimic real-world phenomena: when you walk, you hear footsteps; running blurs your surroundings; and looking at lights disrupts your vision.

We are familiar with this notion as extended reality (XR) or mixed reality (MR) – that is, placing digital objects in the real world and directly interacting with them as if they were actually there. We can use hardware such as head-mounted displays (HMDs) and infrared sensors to augment physical spaces with digital objects and enhance the experience within the space.

However, before we learn more about MR, we must talk about the role of the senses in immersion. Senses are the focal point of our experiences. Without them, we are unable to interpret information or engage with the world around us. If we hear a loud noise, we will cover our ears and try to leave the source of the noise. When we are confronted with the source in the future, our negative experience will inform us how we should respond to that source. Let’s say we were immersed in an environment with a loud noise. That noise provided an unpleasant experience for our ears, and we responded by removing the stimulus. We care about what appeals to our experiences because, in this example, the noise shaped the experience. If the noise wasn’t as loud, it would likely have improved the experience.

Understanding immersion through the senses

We should all be familiar with the major senses: touch, sight, hearing, smell, and taste. We often associate those senses with experience and actions. If you want to taste, you will eat food; if you want to see, you will watch TV; if you want to smell, you will breathe in an aroma; if you want to touch, you will hold something; and if you want to hear, you will listen to music. Those actions toward a stimulus engage the senses and provide us with an experience. Each sense alone provides us with a different way to experience the world around us.

If our senses are tied to our experiences, they become the anchors of an immersive experience. We can say that an immersive experience is something that incorporates multiple senses into one experience. Think of immersion as being on a spectrum, rather than it being all or nothing. You can’t remove immersion completely because it is tied to our senses. Unless you lose all of your ability to sense, you can’t completely remove immersion. Rather, it is on a scale of less immersive to more immersive.

For example, imagine you are walking down a busy street in the middle of rush hour in a major city. You are rushing from a lunch date back to work, so you are holding part of your lunch on your way back. While doing so, someone dumps spoiled milk on you from their balcony. It reeks! In this scenario, you can imagine the type of experience you would have in that situation. Based on the description, you can also isolate each part of that experience into their respective senses:

• Touch: You can feel the food in your hand as you hold your lunch. When the milk lands on you, you feel the liquid on your skin and clothes (fun fact – we can’t feel wetness, but we can feel the difference in pressure and temperature of the liquid compared to the air on our skin).
• Sight: You see tons of cars and people out in the city. Maybe you can even see some buildings and have the sun shining in your eyes.
• Smell: You could be smelling the food you are eating, the smog and sewage of the city, or even the stench of the spoiled milk that was spilled onto you.
• Taste: You could taste the food you had for lunch, or maybe some of the spoiled milk got into your mouth; you probably have a bad taste in your mouth now.
• Hearing: In a busy city, you might hear car horns, engines revving, people talking, and even the sound of the spoiled milk hitting the ground as it also covers you.

When it comes to immersion, you can’t get more immersed than this! The experience I have described incorporates all the senses (Figure 1.1) and would leave you with a vivid and lasting memory:

Figure 1.1 – Human senses that focus on VR include sight, hearing, and touch

When we talk about applications of immersive technology, especially VR, we are in some ways trying to use technology to mimic what we would experience in real life. Research suggests that experiences are more memorable when more senses are tied to those experiences. If you can find ways to build experiences that incorporate the senses in believable ways, the user will feel more engaged, and they will walk away from the experience more informed. We will explore some techniques in future chapters to achieve these goals.

What makes something immersive?

When we talk about being less immersive, we simply mean that we start to remove our senses from the experience. When you remove touch, you can’t feel anything; remove sight, you can’t see the world around you; remove smell, you can’t enjoy aromas; remove hearing and everything is quiet; and remove taste and you cannot enjoy food. The more senses you remove, the less immersive the experience is. Going back to the idea of removing immersion completely, can you do that and still be alive?

Think about the experiences we enjoy and see if you can define what senses are used to make it immersive:

• Reading a book involves seeing and touching
• Listening to music involves listening
• Watching TV involves seeing and listening
• Talking on the phone involves talking and listening
• Swimming involves moving and seeing
• Playing video games involves touching, seeing, and hearing
• Driving a car involves touching, seeing, and hearing

When we talk about these experiences, one crucial component is interaction. We aren’t in stasis: we are acting and reacting to the world around us. Even if the experience is passive, there is a level of interactivity that keeps us engaged. When the experience requires us to perform an action, that makes it interactive. Let’s not confuse interaction with immersion. They are two separate concepts involved in the same experience. You may listen to music, watch a movie, or read a book. Those are passive experiences, but the act of turning the page, changing the channel with a remote, or rewinding a song gives you a level of interaction that keeps you engaged.

To make VR experiences immersive, use the elements of immersion as a guide. We know that immersive experiences are more engaging for users, and the focal point of immersive experiences is our senses. By developing experiences focused on what we see, feel, and do, we can create applications that have a true impact on the VR industry and community. Note that this is independent of the content or industry. These approaches and philosophies can be applied to a variety of industry applications because they speak to the core component of what makes VR different than other mediums such as animation, cinema, or console games. It is the ability to immerse the user in an experience.

How to make experiences immersive

So, how do you make something more immersive? Simple – involve more senses in the experience. Compared to text, video is more immersive because it involves two senses rather than one. Reading incorporates sight and listening incorporates hearing, but watching a movie involves both seeing and listening. To make reading more immersive, add sound. To make listening to music more immersive, add haptic feedback to feel the sound vibrations. When you are thinking about immersion, think about building off the native experience rather than exchanging one element for another. I would not consider video to be the same as immersive reading because you are replacing text with images. Although you are adding sound, you are taking away the text-based visual.

Adding sound to a quiet reading experience such as background music or sound effects can make the reading experience more immersive without taking away the core element of the experience.

With VR, you can take the concept of immersive experiences and build on that framework using technology and digital assets. VR uses interactions in a completely virtual world to let you walk, run, jump, and navigate with motion. Even though you can’t touch the object in the world, haptics can provide limited vibrational feedback. Ultimately, you can see the objects, hear the objects, and orient yourself in spaces among the objects. Compared to being behind a computer screen or gamepad, VR is more immersive because you are in the very location you want to explore, not a proxy of it. You don’t control the character; you are the character.

Now that we have introduced what an immersive experience is and its various components, we can explore what components are essential to achieving such experiences within VR. This will help simplify how to approach building immersive experiences and make the process seem less daunting.