Using SSD technology to support memory overcommitment in vSphere

With applications becoming increasingly memory-hungry, memory always seems to be the resource in the shortest supply on a host. In a virtualized server environment, the lack of sufficient physical memory can have a real impact on project ROI, since it will limit the density that you could otherwise achieve.

That’s because, while CPU sockets can support an ever-increasing number of cores, for maximum efficiency, a host has to have a balance of resources available to virtual machines (VMs). If a resource shortage occurs in any one area on a host -- such as RAM -- the number of VMs that the host can run will be restricted despite plentiful resources in other areas.

The technique of memory overcommitment in a virtualized environment can help mitigate the problem, and using SSD to handle memory overcommitment can produce better results at a lower cost than memory overcommitment on mechanical disk.

Let’s examine the memory limitations of server hardware, how memory overcommitment works, and how solid-state drive (SSD) technology can address the problem.

The constraints of physical memory in server hardware

There are two challenges when it comes to memory in a server: the number of available memory slots and the cost of higher-density dual-inline memory modules (DIMMs). Server hardware used in virtualization requires a great deal of memory to support the many VMs that will be running on it. The latest version of vSphere can support up to 1 TB of memory in a host and up to 255 GB of memory in a virtual machine. Having a limited number of memory slots in a server restricts the amount of RAM that you can put in the server and requires the use of high-density DIMMs, which can be very expensive. Even if you have plentiful memory slots available, if you want to have the most memory possible, you need to use DIMM sizes greater than 4 GB. The price of 8 GB or 16 GB DIMMs for servers can range from $400 to $1,300 per DIMM depending on speed, rank and other factors. As a result, configuring a virtual host with 128 GB of memory can cost from $6,000 to $20,000.

vSphere (but not Hyper-V) has the ability to overcommit memory, enabling you to provision more memory to VMware VMs than a host physically has present. This can keep memory costs down by allowing you to use less memory in a host. It is also useful when a host has reached its physical memory limit and has had all its DIMM slots populated with the largest DIMM size available. With memory overcommitment, a VM can be configured with up to 255 GB of memory even if a host only has 8 GB of memory. The host makes up for the extra memory that it does not physically have by utilizing a special virtual swap (.vswp) file that is created on disk when a VM is powered on. A .vswp file is created equal in size to the amount of memory assigned to a VM minus any memory reservations (the default is 0) that a VM may have set on it (for example, a 16 GB VM with a 4 GB reservation will have a 12 GB .vswp file created). These files are always created for virtual machines but only used if a host exhausts all of its physical memory.

While overcommitting memory is a nice feature, it comes with a fundamental problem: Mechanical disk drives are drastically slower than RAM is. As a result, a VM that is forced to use disk as memory will experience greatly reduced performance -- a situation that you want to avoid in most cases. Depending on the amount of other disk activity, a VM could become so slow that it becomes unresponsive when swapping to disk.

Solving the overcommitment performance problem with SSD technology

SSDs offer a creative solution to greatly reduce the performance penalty that is incurred when swapping to disk. A mechanical 15,000 rpm SAS drive has a typical throughput of about 180 IOPS; the throughput of an SSD is much greater, with a typical 3 Gbps SATA SSD drive able to handle at least 5,000 IOPS for writes and 20,000 IOPS for reads. Using an SSD as the storage device for virtual machine swap files allows you to make use of memory overcommitment without taking a big hit in performance. While SSDs do not offer the same performance as RAM, they are a big improvement over mechanical drives.

SSDs can essentially become another tier of RAM in a server that can be used by virtual machines. Memory resource controls in vSphere can be used to guarantee or limit the amount of physical memory that VMs can use so you can effectively control when VMs swap to disk. Setting a memory reservation will guarantee that a VM will always have a specific amount of physical memory allocated to it; setting a memory limit will restrict the amount of physical memory a VM can use. This enables you to ensure that critical VMs have access to faster physical memory and that the SSD RAM tier is used by less important VMs once physical memory is exhausted.

Cost comparison: RAM vs. SSD

To illustrate the cost proposition of using SSD for memory overcommitment, let’s look at a pricing example for two configurations of an HP server -- the first with 192 GB of RAM and a traditional hard disk and the second with 64 GB of RAM and an SSD.

• Configuration 1: HP DL385 G7 with dual 12-core CPUs and 192 GB of memory, 72 GB 15,000 rpm SAS drive; cost: $15,065
• Configuration 2: HP DL385 G7 with dual 12-core CPUs and 64 GB of memory, 120 GB “midline” drive; cost: $11,141

The first configuration provides you with 192 GB of physical RAM; the 72 GB disk isn’t very usable for virtual swap files because of its slow speed. The second configuration provides you with 64 GB of physical RAM plus another 120 GB of SSD storage that can be used for virtual swap files. Current SSD prices from HP are about $1,000 for a 60 GB drive and $2,000 for a 120 GB drive. The net result is that using an SSD as a substitute for RAM would cost almost $4,000 less in this configuration. With SSD prices steadily dropping and their capacities increasing, the savings and benefits of using SSDs will be even greater as time goes on.

Instead of replacing RAM with SSD storage, you can also use SSDs to supplement RAM. The HP server used in the example above has a total of 24 CPU cores and could easily run 50 or more VMs. But, the maximum memory that the server can support is only 256 GB. You could add SSD drives to provide additional usable memory to the VMs to increase the density of the host to fully utilize the CPU resources in the server.

Whether they are used as a memory tier or a storage tier, SSD technology provides a huge performance boost over mechanical hard drives, providing you with more options when architecting servers to use with virtualization. Using SSDs in lieu of RAM is just one of the many creative ways that you can leverage SSDs in your virtual environment. They provide more capacity per dollar than RAM does and can be used in any sized virtualization solution to provide additional memory capacity. Virtualization requires that the resources of a host be balanced to prevent bottlenecks from occurring and also to achieve maximum efficiency of the expensive server hardware used for virtualization. SSDs offer a way to achieve that balance and can  help provide your memory-hungry VMs with the resources they need.