Increasing network bandwidth available to virtual machines

A big server virtualization trend involves achieving the maximum virtual machine (VM) density within the available host servers. As the VM density increases, the hardware cost per VM decreases, and network administrators will often pack physical servers with as many processor cores and as much memory as the server can accommodate. Unfortunately, the server’s network ports often limit the server’s VM density as each VM on the host server requires network connectivity. While a network interface card (NIC) can be shared among multiple VMs, there is only so much network bandwidth to go around. This can be problematic, especially if some of the virtual servers are running network-intensive applications.

Fortunately, there are ways to increase the amount of network bandwidth available to your VMs, possibly allowing you to increase the VM density on your host servers.

Using additional network hardware
The easiest solution is to install additional NICs in your host server. The physical hardware only provides one or two integrated network ports on the motherboard and only has a limited number of expansion slots. All expansion slots could be in use, but you may still be able to increase the number of network ports installed in your server by using NICs that have multiple ports. For example, some PCI-X and PCI Express (PCIe) cards contain four separate network ports on a single card. Using these cards can quadruple the number of network ports installed in your server.

Another option may be to take advantage of external NICs. For example, if you run out of network ports on your host server, you could add one or more USB network adapters. USB-based network adapters, however, are not always ideal. Some virtualization platforms will not expose USB devices to VMs. Even if this is the case, you may still benefit by adding a USB network adapter.

A best practice is to reserve a NIC for management traffic. You can't dedicate all of the available network bandwidth solely to VMs -- there must be a way to communicate with the parent partition (machine) to manage it. Even if your virtualization software exposes a USB NIC to the VM, you may be able to dedicate a USB NIC to the parent partition. If you had previously dedicated one of the other NICs into your system to management traffic, then using a USB NIC instead will free that NIC for use with your VMs.

It’s usually possible to share a single NIC among multiple VMs. The problem is that a single NIC provides a limited amount of network bandwidth, and bandwidth must be shared among all VMs bound to the NIC. But you can increase the available network bandwidth by installing faster NICs instead of increasing the quantity of NICs installed in your server. For example, switching to 10 Gigabit Ethernet can make the difference in virtual server performance.

Introduce virtual networks
If there isn’t a practical way to add additional network bandwidth to your server, there is one more trick to try:  decreasing the demands on your existing bandwidth instead of increasing the server's total network bandwidth.

One way to accomplish this is by creating a virtual network segment, a network segment that exists solely within the host server. If you’re using Microsoft Hyper-V or VMware, there already is at least one virtual network. Every physical network adapter used by a VM is connected to a virtual switch. That virtual switch is in turn connected to a virtual network adapter on each of the VMs that use the physical NIC.

Virtual networks commonly provide VMs with connectivity to the physical network, but they don't have to access the physical network. It’s possible to create a virtual network segment that services VMs, but doesn’t tie into the physical network. You could create such a segment to offload some of the network traffic from your physical network.

For example, if you had a virtualized Web server that frequently used a back-end SQL Server database running on another virtual server, you could ensure that database queries don’t consume physical network bandwidth by creating a dedicated virtual network segment between the Web and database servers. By doing so, the network bandwidth available to your virtual servers is increased even without adding additional network ports.

Match network port use to server needs
Creating additional virtual network segments will only help if you can offload a significant amount of network traffic to the virtual network segment. If this isn’t possible, by dedicating certain adapters to certain servers, you may still be able to take advantage of how your virtualization platform uses virtual switches for each physical network adapter.

The idea is that some servers inevitably consume more network bandwidth than others. Since you probably don't have enough network ports to dedicate a port to every virtual server, you can decide how NICs should be shared based on each server's bandwidth consumption. For example, if you have a SQL Server that consumes a lot of network bandwidth, then you could dedicate a NIC to it. Conversely, you may have domain controllers and a DHCP server that don’t use much bandwidth. As such, you could probably get away with sharing a single NIC among these servers.

The availability of network bandwidth has the potential to limit a host server's overall VM density, but there are several ways to increase the amount of bandwidth available to the server and make more efficient use of your server's existing bandwidth.

ABOUT THE AUTHOR: Brien M. Posey has received Microsoft’s Most Valuable Professional award six times for his work with Windows Server, IIS, file systems/storage and Exchange Server. He has served as CIO for a nationwide chain of hospitals and healthcare facilities and was once a network administrator for Fort Knox.