Take the sting out of switching cloud data storage providers

Qualify potential new providers

What's involved in moving data and service contracts? If you're storing large amounts of data in the cloud (hundreds of terabytes or even petabytes), the process can be challenging and time-consuming. Before embarking, it's important to ensure the new provider's services and capabilities accommodate your data storage needs. For example, when migrating unstructured data, check that it supports the file protocols or object storage access APIs you're using or planning to use. Do its service-level agreement (SLA) terms support your data availability requirements? How will overall costs compare? Take advantage of cost calculators and other resources from your current and potential new providers to determine how much more or less you'll be paying each month.

If you're also moving apps, check on the portability of the virtual machine (VM) images or app containers you're running, and decide whether they can simply be migrated or will need rebuilding. Also, qualify whether the new provider meets your application performance objectives and other critical requirements (e.g., for network latency or total throughput) by doing a small-scale trial or proof-of-concept. Though app migration is beyond the scope of this article, we bring it up because you'll need to include it when planning if you're currently running or intend to run workloads in the cloud.

Choose a migration approach

Once you're comfortable that a new provider can address your needs, start scoping your migration effort. You'll need to answer some basic questions:

• How much data will migrate?
• How long will it take given the options available?
• Will users require access to data during the migration? If yes, how will you ensure access while managing costs?
• What recourse or contingency plan can you implement should migration stall or the new provider's services and capabilities fall short of what you expected?

The answers determine how and when to migrate. Then, evaluate the available migration options of the new provider, which generally boil down to the following:

1. Options should include online migration of encrypted data over the public internet using a "brute-force" approach such as Secure FTP (SSH-2 based) or similar technology.
2. They should allow online transfers using virtual private network (VPN) tunneling via a secure private connection.
3. They should provide transport of bulk data via physical media (e.g., portable storage devices).
4. And they should include transfer via a provider appliance, third-party tool or service.

While most users choose to employ one of these methods, you can certainly use more than one approach if it makes sense.

The first approach works best when transferring relatively small data sets, since it's subject to the congestion and unpredictable latencies of the public internet and may consume lots of bandwidth. As a result, data transfers may take days or weeks to complete, even for a few terabytes. Remember, while data transfer costs may be reasonable, your current provider will still charge for outbound data transfer.

The second option lets you migrate rapidly, but can be costly when you migrate large amounts of data. This approach requires working with major public cloud data storage providers, such as AWS or Microsoft Azure, that have a relationship with the same network or data center services provider, such as Equinix. You would directly connect with the network services provider, which in turn can provide a VPN tunnel connection to your cloud storage accounts. Data would then flow securely from your existing cloud provider through the network services vendor and back out to the new provider's service.

The major advantage here is you can set up secure and reliable connections that aren't subject to the vagaries of the public internet for data transfer between providers. The major downside is the cost of the VPN tunnel services, which can be expensive. For example, AWS Direct Connect VPN tunnel service charges both an hourly charge per port (based on speed) and per-gigabyte egress fee, while Azure ExpressRoute service charges a monthly fee based on port speed. If you need to transfer critical data quickly, the costs could be worth it, but in most cases, it wouldn't be cost-effective.

Use the third alternative when planning to transfer large amounts of data to a new provider, but uploading or downloading over the network is unattractive due to limited bandwidth or high costs. You first transfer data from an existing cloud storage service, typically by sending the provider an HDD and creating an export job online specifying what data to transfer. The provider then moves your data, encrypts it and returns the HDD to you. You then ship it to your new provider for upload. Sometimes, you can streamline the transfer by specifying the HDD be shipped directly to the new provider. This process sounds inefficient, but it's generally reliable. Azure Import/Export and AWS Import/Export Disk are examples of transfer services that utilize physical media.

Cloud data storage providers clearly aren't motivated to ease data migration to other providers. As a result, you'll probably find the process more cumbersome and costly than expected. This has led a number of third-party vendors to introduce tools and services that facilitate migration efforts (see "Third parties that help with cloud migration"). In addition to these third-party options, some cloud providers also offer file sync and sharing or replication and recovery services that ease data transfer to servers in your data center, where files can, in turn, be migrated to a new cloud storage provider. When in doubt, talk to both your current and destination providers to understand other available tools and services that might help simplify or expedite your migration effort. 

Avoid cloud lock-in and ease migration

What can you do to avoid getting stuck with a cloud provider? We believe the following guidelines will help:

Vigorously qualify your short list of providers upfront to reduce the risk of having to switch later. As part of due diligence, determine whether they offer the cloud data uses you're interested in today or might expand to them in the future. Can they accommodate your requirements for availability, performance, security and other key SLA metrics? Carefully review the SLA terms and spend a month conducting a trial assessment. Are costs reasonable for the selected uses? Reach out to peers for their experiences with providers.

Consider how portable your cloud data and apps would be should you need to change providers. Will data be stored in a format that'll allow for easy movement down the road? Similarly, are the VM images or application containers you're running relatively portable? Most cloud data storage providers run customized hypervisors (based on Xen, KVM and so on), which means images may need to be modified before transfer. Converting to an open VM format (like open virtualization format) may be helpful, but the overall migration process will require some effort when running lots of VMs. Containers tend to be much more portable, but few enterprises have more than a handful of apps running in containers today.

Check out the provider's tools, services and associated costs for moving data out of the cloud. Would these enable you to cost effectively migrate assets to new providers? Are third-party tools and services available to fill any gaps? Think through the likely migration process, and determine how much IT staff time and other resources may need to be invested. Can you afford to take on a cloud-to-cloud migration project given the amount of data you'll be storing with your new provider?

If storing primary data is critical to your business in the cloud, consider how you might continue to provide access during migration. For example, you may want to stage the migration effort, replicating or accelerating the transfer of more important data to the new cloud, or do it off hours or at times of lower demand.

Tier data via a cloud storage gateway or similar platform to avoid getting stuck with critical business data in the cloud that's difficult to move. With this hybrid storage approach, primary and active (transaction-oriented) data is stored in your data center, while less active or inactive (secondary or tertiary) data may be stored in the cloud. Tiering can help avoid the unpleasant and costly prospect of migrating "warm" or "hot" data between providers.

About the author:
Jeff Byrne is a senior analyst and consultant at Taneja Group. He can be reached at [email protected].