Fighting guest CPU saturation in SMP VMs

Guest CPU saturation happens when the application and operating system running in a VM use all of the CPU resources that the ESXi host is providing for that VM. However, this guest CPU saturation does not necessarily indicate that a performance problem exists.

Compute-intensive applications commonly use all of the available CPU resources, but this is expected and might be acceptable (as long as the end user thinks that the job is completing quickly enough). Even less-intensive applications might experience periods of high CPU demand without experiencing performance problems. However, if a performance problem exists when guest CPU saturation is occurring, steps should be taken to eliminate the condition.

When a VM is configured with more than one vCPU but actively uses only one of the vCPUs, resources that could be used to perform useful work are being wasted. At this time, you may at least see a potential performance problem from the most active vCPU perspective.

Getting ready

To step through this recipe, you need a running ESXi Server, a couple of running CPU-hungry VMs, a vCenter Server, and vSphere Web Client. No other prerequisites are required.

How to do it...

To spot CPU overcommitment in the guest OS, ready time and usage percentage are two CPU resource parameters that you should monitor closely:

1. Log in to vCenter Server using vSphere Web Client.
2. On the home screen, navigate to Hosts and Clusters.
3. Expand the ESXi host and go to the CPU-hungry VM.
4. Navigate to the Monitor tab.
5. Navigate to the Performance tab.
6. Navigate to the Advanced view.
7. Click on Chart Options.

 

8. Navigate to CPU from Chart metrics.
9. Navigate to the VM object.
10. Navigate to the Advanced tab and click on Chart Options.
11. Select only Usage, Ready, and Used in the Counters section and click on OK:

The preceding example shows the high usage and used value on a VM configured with one vCPU. We can see its overall CPU usage is 100 percent:

The preceding example shows that after you added a second vCPU to the VM, the percentage of overall CPU usage dropped down to 52 percent.

How it works...

So in the case of an SMP VM, if it demands high CPU resources, it may happen that either the application is single threaded, or the guest operating system is configured with uniprocessor HAL.

Many applications are written with only a single thread of control. These applications cannot take advantage of more than one processor core.

In order for a VM to take advantage of multiple vCPUs, the guest operating system running on the VM must be able to recognize and use multiple processor cores. If the VM is doing all of its work on vCPU0, the guest operating system might be configured with a kernel or HAL that can recognize only a single processor core.

In the preceding graph, the OS is sharing the load of the single-threaded application between both the available vCPUs.

You have two possible approaches to solve performance problems related to guest CPU saturation:

• Increase the CPU resources provided to the application
• Increase the efficiency with which the VM uses CPU resources

Adding CPU resources is often the easiest choice, particularly in a virtualized environment. If a VM continues to experience CPU saturation even after adding CPU resources, the tuning and behavior of the application and operating system should be investigated.