Storage vMotion: Application and performance

Storage vMotion, a feature that was first introduced in VMware ESX 3.5, allows you to migrate the virtual disks of a running virtual machine from one data store to another on the same host.

While at first glance Storage vMotion seems similar to vMotion proper -- they both move running VMs -- there is a distinct difference between the two. vMotion moves a VM from one host to another but retains the same storage location of the VM. Storage vMotion, on the other hand, changes the storage location of the VM on the same host. A VM can be moved to any data store on the same host as long as that host is connected to both the source and target data store.

How does Storage vMotion work?

When a Storage vMotion is initiated, the following process occurs:

1.      A new virtual machine directory is created on the target data store; virtual machine configuration files and all non-virtual disk files are copied to the target directory.

2.      A shadow VM is started on the target data store using the copied files; it then sits idle as it waits for the virtual disk copy to be completed.

3.      Leveraging the Changed Block Tracking (CBT) feature in vSphere, an initial copy of the VM’s virtual disk files is made to the target data store; any disk changes that occur during the copy are tracked by the CBT feature. Data is not copied over the vMotion network. Instead, a special high-performance DataMover module of the VMkernel is used. (In ESX 3.5, a snapshot was taken of the virtual machine disks, so they were read-only during the copy and changes were kept in a delta file.)

4.      During the virtual disk copy, any changed blocks are continually copied from the source data store to the target data store. After the virtual disk copy completes and the number of changed blocks can be copied in less than five seconds, a self-vMotion is performed to transfer the running VM over to the idle shadow VM.

5.      The disk files and directory from the source data store are deleted.

Storage vMotion is a safe process; there’s no risk of downtime or data loss while it occurs. The target data store is checked for adequate free space before the copy is started; it fails if there is not enough available space. If something occurs while the copy is in progress that results in insufficient disk space on the target data store (for instance, if a new VM is created), the Storage vMotion process fails, the files copied to the target datastore are deleted, and the VM continues to run on the source data store. If the VM has very high disk I/O occurring while a Storage vMotion process is in progress and the changed blocks cannot be copied fast enough to the target data store, the Storage vMotion process will eventually fail and will fall back to the source.

What’s Storage vMotion used for?

Storage vMotion has many uses; it can be an invaluable tool when performing storage maintenance as VMs can be easily moved to other storage devices while they are running so a storage device can be shut down. They can then be moved back once maintenance on the storage device has been completed.

But Storage vMotion can be used for more than just moving disk files from one to data store to another; it can also be used to change the disk format of a VM. When a disk is moved, you can choose to convert thick disks that are fully allocated to thin disks, and vice versa, as part of the copy process. You also have the option of choosing the same data store for the source and target so the disk format can be changed without actually moving the disk to another data store. Storage vMotion can also be used to re-shrink a thin disk after it has grown in size and data has been deleted from it. However, deleted disk blocks must first be zeroed from within the guest OS using a utility like Microsoft’s sdelete.

One other handy use for Storage vMotion is for renaming virtual disk files. When a VM is renamed in vCenter Server, the names of the corresponding VM directory and files are not changed, which can cause confusion. Applying the Storage vMotion process on a VM will automatically align the VM directory and file names with the name of the VM so they are again the same.

What system requirements and limitations does Storage vMotion have?

Storage vMotion is a licensed feature and is only available in the Enterprise and Enterprise Plus editions of vSphere. There are not many requirements for using Storage vMotion except that a VM cannot have an active snapshot.

A Storage vMotion can be very resource-intensive on the host and storage device, and there are limits on how many you can perform at the same time. In vSphere 4.1, you can perform up to two Storage vMotions simultaneously on the same host or up to eight on the same data store. In vSphere 4.1, a new hardware offload DataMover was introduced that leverages the vStorage APIs for Array Integration (VAAI), which allows the process to be offloaded to a storage array that supports VAAI. This has two benefits: The copy stays within the storage array so there is less resource usage on the host, and it makes the copy operation much faster. 

Unlike vMotion, which requires a VMkernel port to be configured on a vSwitch before it can be used, there is nothing to install or configure to use Storage vMotion.

What are the steps for performing a Storage vMotion?

The Storage vMotion process is initiated in the vSphere client by selecting a VM and choosing the Migrate option; once the Migrate VM wizard loads, you have three options to choose from:

• Change Host: This performs a vMotion.
• Change Datastore: This applies the Storage vMotion process.
• Change Host and Datastore: This performs a cold migration (the VM must be powered off).

After you select Change Datastore, you choose a destination data store, which can be the same for disk format changes, and a disk format  (the choices are Same, Thick or Thin), and then the operation will begin. Besides using the vSphere client, you can also use the svmotion.pl command to perform a Storage vMotion using the vSphere command-line interface (CLI) from a workstation or with the vSphere Management Assistant (vMA).

The combination of vMotion and Storage vMotion in vSphere provides great flexibility to move powered-on VMs between hosts or data stores as needed. This allows you to move workloads around so they are more balanced or to avoid downtime when maintenance needs to be performed. Storage vMotion is one of those features that you may not end up using very often, but when you do use it, you’ll really appreciate it.

Eric Siebert is a VMware expert and author of two books on virtualization.