Benefits and technologies of virtualization

If you are already familiar with virtualization, this chapter will provide a review of many of the benefits and technologies of virtualization.

Since the focus of this book is on design, we will not go into great detail discussing the specifics of how to configure resources in a virtual data center. Most of you probably already have a good understanding of VMware's virtualization architecture, so this chapter will just provide a basic overview of the key VMware components that are the building blocks to the virtual data center.

Virtualization creates a layer of abstraction between the physical hardware and the virtual machines that run on it. Virtual hardware is presented to the virtual machine granting access to the underlying physical hardware, which is scheduled by the hypervisor's kernel. The hypervisor separates the physical hardware from the virtual machine, as shown in the following diagram:

Logical representation of hypervisor layer

The hypervisor separates the physical hardware from the virtual machines. The new release of vSphere 6.7 does not change the design process or the design methodologies. The new functions and features of the release provide an architect with more tools to satisfy design requirements.

The hypervisor

At the core of any virtualization platform is the hypervisor. The VMware hypervisor is named vSphere ESXi, simply referred to as ESXi. ESXi is a Type 1 or bare-metal hypervisor. This means that it runs directly on the host's hardware to present virtual hardware to the virtual machines. In turn, the hypervisor schedules access to the physical hardware of the hosts.

ESXi allows multiple virtual machines with a variety of operating systems to run simultaneously, sharing the resources of the underlying physical hardware. Access to physical resources, such as memory, CPU, storage, and network, used by the virtual machines is managed by the scheduler, or Virtual Machine Monitor (VMM), provided by ESXi. The resources presented to the virtual machines can be over committed; this means more resources that are physically available can be allocated to the virtual machines on the physical hardware. Advanced memory sharing and reclamation techniques, such as Transparent Page Sharing (TPS) and ballooning, along with CPU scheduling, allow for over commitment of these resources to be possible, resulting in greater virtual-to-physical consolidation ratios.

ESXi 6.7 is a 64-bit hypervisor that must be run on a 64-bit hardware. An ESXi 6.7 installation requires at least 1 GB of disk space for installation. It can be installed on a hard disk locally, a USB device, a Logical Unit Number (LUN) on a Storage Area Network (SAN), or deployed stateless on hosts with no storage using Auto Deploy. The small footprint of an ESXi installation provides a reduction in the management overhead associated with patching and security hardening.

With the release of vSphere 5.0, VMware retired the ESX hypervisor. ESX had a separate, Linux-based service console for the management interface of the hypervisor. Management functions were provided by agents running in the service console. The service console has since been removed from ESXi, and agents now run directly on ESXi's VMkernel.

To manage a standalone host running ESXi, a Direct Console User Interface (DCUI) is provided for basic configuration and troubleshooting. A shell is available that can either be accessed locally from the console or remotely using Secure Shell (SSH). The esxcli command-line tools and others can be used in the shell to provide advanced configuration options. An ESXi host can also be accessed directly using the vSphere Client. The ESXi DCUI is shown in the following screenshot:

Screenshot of ESXi's DCUI

The DCUI can be accessed remotely using SSH by typing the dcui command in the prompt. Press Ctrl + C to exit the remote DCUI session.

Virtual machines

A virtual machine is a software computer that runs a guest operating system. Virtual machines are comprised of a set of configuration files and data files stored on local or remote storage. These configuration files contain information about the virtual hardware presented to the virtual machine. This virtual hardware includes the CPU, RAM, disk controllers, removable devices, and so on, and emulates the same functionality as the physical hardware. The following screenshot depicts the virtual machine files that are stored on a shared Network File System (NFS) datastore:

Virtual machine files stored on a shared NFS datastore displayed using the vSphere Web Client

The files that make up a virtual machine are typically stored in a directory set aside for the particular virtual machine they represent. These files include the configuration file, virtual disk files, NVRAM file, and virtual machine log files.

The following table lists the common virtual machine file extensions along with a description of each:

File extension	Description
.vmx	This is a virtual machine configuration file. It contains the configurations of the virtual hardware that is presented to the virtual machine.
.vmdk	This is a virtual disk descriptor file. It contains a header and other information pertaining to the virtual disk.
-flat.vmdk	This is a preallocated virtual disk. It contains the content or data on the disk used by the virtual machine.
.nvram	This is a file that stores the state of a virtual machine's Basic Input Output System (BIOS) or Extensible Firmware Interface (EFI) configurations.
.vswp	This is a virtual machine swap file. It gets created when a virtual machine is powered on. The size of this file is equal to the amount of memory allocated minus any memory reservations.
.log	This is a virtual machine log file.
.vmsd	This is a virtual machine file used with snapshots to store data about each snapshot active on a virtual machine.
.vmsn	This is a virtual machine snapshot data file.

Virtual machines can be deployed using a variety of methods, as follows:

• Using the New Virtual Machine Wizard in the vSphere Client or vSphere Web Client
• By getting converted from a physical machine using the VMware Converter
• By getting imported from an Open Virtualization Format (OVF) or Open Virtualization Archive (OVA)
• By getting cloned from an existing virtual machine
• By getting deployed from a virtual machine template

When a new virtual machine is created, a guest operating system can be installed on the virtual machine. VMware vSphere 6.7 supports more than 120 different guest operating systems. These include many versions of the Windows server and desktop operating systems, many distributions and versions of Linux and Unix operating systems, and Apple macOS operating systems.

Virtual appliances are preconfigured virtual machines that can be imported to the virtual environment. A virtual appliance can be comprised of a single virtual machine or a group of virtual machines with all the components required to support an application. The virtual machines in a virtual appliance are preloaded with guest operating systems, and the applications they run are normally preconfigured and optimized to run in a virtual environment.

Since virtual machines are just a collection of files on a disk, they become portable. Virtual machines can be easily moved from one location to another by simply moving or copying the associated files. Using VMware vSphere features, such as vMotion, Enhanced vMotion, or Storage vMotion, virtual machines can be migrated from host to host or datastore to datastore while a virtual machine is running. Virtual machines can also be exported to an OVF or OVA to be imported into another VMware vSphere environment.

Virtual infrastructure management

VMware vCenter Server provides a centralized management interface to manage and configure groups of ESXi hosts in the virtualized data center. The vCenter Server is required to configure and control many advanced features, such as the Distributed Resource Scheduler (DRS), Storage DRS, and VMware High Availability (HA). The vCenter Server management Graphical User Interface (GUI) is accessed using the browser-based vSphere Client. Many vendors provide plugins that can be installed to allow third-party storage, network, and compute resources to be managed using the vSphere Client.

vCenter access using the C#, or Windows vSphere Client, is only available in versions prior to 6.5. Since the release of vSphere 5.5, however, access to, and the configuration of, new features is only available using the vSphere Web Client. The vSphere Web Client can be accessed at https://FQDN_or_IP_of_vCenter_Server:9443/.

vCenter Server 6.7 must use a 64-bit architecture if installed on a Windows Server. It can be run on dedicated physical hardware or as a virtual machine. When the vCenter Server is deployed on Windows, it requires either the embedded PostgreSQL database, a Microsoft SQL database, or an Oracle database to store configuration and performance information. IBM DB2 databases are supported with vSphere 5.1, but this support was removed in vSphere 5.5.

With the release of vCenter 6.0, the Microsoft SQL Express database is no longer used as the embedded database. Embedded PostgreSQL is now used as the embedded database for small deployments. The PostgreSQL database on a Windows Server can be used to support environments of less than 20 hosts and 200 virtual machines. When upgrading to vCenter 6.7, if the previous version was using the Microsoft SQL Express database, the database will be converted to the embedded PostgreSQL as part of the upgrade. The embedded PostgreSQL database is suitable for almost all deployments, but using an external database is still supported.

Another option for deploying the vCenter Server is the vCenter Server Appliance (VCSA). The VCSA is a preconfigured, Linux-based virtual machine preinstalled with the vCenter Server components. The appliance includes an embedded PostgreSQL database that supports the configuration maximums of 2,000 hosts and 25,000 powered-on virtual machines.

Several other management and automation tools are available to aid the day-to-day administration of a vSphere environment: the vSphere Command-Line Interface (vCLI); vSphere PowerCLI provides a Windows PowerShell interface; vRealize Orchestrator can be used to automate tasks; and the vSphere Management Assistant (vMA) is a Linux-based virtual appliance that is used to run management and automation scripts against hosts. vMA was deprecated, and its final release only supports vSphere 6.5. These tools allow an administrator to use command-line utilities to manage hosts from remote workstations.

VMware provides a suite of other products that benefit the virtualized data center. These data center products, such as VMware vRealize Operations (vROps), VMware Site Recovery Manager (SRM), and VMware vRealize Automation (vRA), can each be leveraged in the virtual data center to meet specific requirements related to management, disaster recovery, and cloud services. At the core of these products is the vSphere suite, which includes ESXi, the vCenter Server, and the core supporting components.

Understanding the benefits of virtualization

The following table provides a matrix of some of the core VMware technologies and the benefits that can be realized by using them:

VMware technology	Primary benefits	Description
vSphere ESXi	Server consolidation Resource efficiency	ESXi is VMware's bare-metal hypervisor that hosts virtual machines, also known as guests, and schedules virtual hardware access to physical resources.
vSphere HA	Increased availability	HA restarts virtual machines in the event of a host failure. It also monitors and restarts the virtual machines in the event of a guest operating system failure.
vMotion and vSphere DRS	Resource efficiency Increased availability	vMotion allows virtual machines to be live-migrated between hosts in a virtual data center. DRS determines the initial placement of the virtual machine on the host resources within a cluster and makes recommendations, or automatically migrates the virtual machines to balance resources across all hosts in a cluster.
Resource pools	Resource efficiency	These are used to guarantee, reserve, or limit the virtual machine's CPU, memory, and disk resources.
VMware Fault Tolerance (FT)	Increased availability	FT provides 100 percent uptime for a virtual machine in the event of a host hardware failure. It creates a secondary virtual machine that mirrors all the operations of the primary. In the event of a hardware failure, the secondary virtual machine becomes the primary and a new secondary is created.
Thin provisioning	Resource efficiency	This allows for storage to be over provisioned by presenting the configured space to a virtual machine, but only consuming the space on the disk that the guest actually requires.
Hot add CPU and memory	Resource efficiency scalability	This allows for the addition of CPU and memory resources to a virtual machine while the virtual machine is running.
Storage vMotion	Resource efficiency	This moves virtual machine configuration files and disks between storage locations that have been presented to a host.
vSphere Storage Application Programming Interface (APIs); data protection	VM backups and disaster recovery	Allows third parties to build agentless backup and disaster recovery solutions that integrate with the vSphere platform
vSphere replication	Disaster recovery	This features provides the ability to replicate virtual machines between sites.
vCenter server	Simplified management	This provides a single management interface to configure and monitor the resources available to virtual data centers.
vCenter server linked mode	Simplified management	This links multiple vCenter Servers together to allow them to be managed from a single client.
Host profiles	Simplified management	This maintains consistent configuration and configuration compliance across all the hosts in the environment.

This is not meant to be an exhaustive list of all VMware technologies and features, but it does provide an insight into many of the technologies commonly deployed in the enterprise virtual data center.

There are many others, and each technology or feature may also have its own set of requirements that must be met in order to be implemented. The purpose here is to show how features or technologies can be mapped to benefits that can then be mapped to requirements and ultimately mapped into a design. This is helpful in ensuring that the benefits and technologies that virtualization provides satisfy design requirements.

Identifying when not to virtualize

Not all applications or server workloads are good candidates for virtualization. It is important that these workloads are identified early on in the design process.

There are a number of reasons why a server or application may not be suitable for virtualization. Some of these include the following:

• Vendor support
• Licensing issues
• Specialized hardware dependencies
• High resource demand
• Lack of knowledge or skillsets

A common reason to not virtualize an application or workload is the reluctance of a vendor to support their application in a virtual environment. As virtualization has become more common in the enterprise data center, this has become uncommon; but, there are still application vendors that will not support their products once virtualized.

Software and operating system licensing in a virtual environment can also be a challenge, especially when it comes to physical server to virtual machine conversions. Many physical servers are purchased with Original Equipment Manufacturer (OEM) licenses, and these licenses, in most cases, cannot be transferred to a virtual environment. Also, many licenses are tied to hardware-specific information, such as interface MAC addresses or drive signatures. Licensing issues can usually be overcome. Many times, the primary risk becomes the cost to upgrade or acquire new licensing. As with other potential design risks, it is important that any issues and potential impacts licensing may have on the design be identified early on in the design process.

Some applications may require the use of specialized hardware. Fax boards, serial ports, and security dongles are common examples. There are ways to provide solutions for many of these, but often, given the risks associated with the ability to support the application, or the loss of one or more of the potential benefits of virtualizing the application, the better solution may be to leave the application on dedicated physical hardware. Again, it is important that these types of applications be identified very early on in the design process.

Physical servers configured with a large amount of CPU and memory resources where applications are consuming a large amount of these resources may not be good candidates for virtualization. This also holds true for applications with high network utilization and large storage I/O requirements. vSphere 6.7 supports virtual machines configured with up to 128 virtual CPUs (vCPUs) and 6 TB of memory, but the high utilization of these configured resources can have a negative impact on other workloads in the virtual environment. These high-utilization workloads will also require more resources to be reserved for failover. The benefits of virtualizing resource-intensive applications must be weighed against the impact placed on the virtual environment. In some cases, it may be better to leave these applications on dedicated physical hardware.

Many administrators may lack knowledge of the benefits or skills to manage a virtualized data center. The administrator of a virtual environment must be well-versed with storage, networking, and virtualization in order to successfully configure, maintain, and monitor a virtual environment. Though this may not necessarily be a reason not to leverage the benefits of a virtualized environment, it can be a substantial risk to the acceptance of a design and the implementation. This is especially true with smaller IT departments, where the roles of the server, application, storage, and network administrators are combined.

Each of these can introduce risks in the design. We will discuss how risk impacts the design process in much more detail in Chapter 2, The Discovery Process, and Chapter 3, The Design Factors.