What you will learn in this tip: As your company grows, there's a good chance you'll eventually outgrow your current storage environment. SAN data migration projects are labor-intensive and complicated. But with proper planning and our data migration checklist, you can ensure your project doesn't take more time and money than planned.
There's a common impression that data migration involves simply moving data from the old storage array to the new. That impression is incorrect. Moving the data is just a small part of data migration and generally the easiest part.
A storage data migration is a complicated series of manually intensive data management tasks. In fact, there are 43 discrete tasks, according to SANpulse Technologies, a company that specialized in storage migration. A mere six of those tasks are semiautomated, and 37 are labor-intensive. They range from preplanning analysis to running the migration to deactivating legacy systems to validating the migration, verifying data integrity and doing final cleanup and documentation.
Many data migration tasks are repetitive actions. Take, for example, data discovery and collection for physical and virtual servers. All of the processes must be repeated for each physical and virtual server. But it's server remediation -- making sure it will boot up; is up to date; has the correct drivers, microcodes and storage adapters; works with the new SAN; has the correct pathing, etc. -- that takes huge swaths of time and effort. And such remediation must be done for each and every physical and virtual server. That's a lot of steps for just one task.
SAN data migration checklist
Click the image to download
the checklist.
What follows are the 43 tasks SANpulse developed to do a complete and thorough SAN data migration. The tasks are divided among six steps to make it easier to organize and plan the work. All of the tasks are bulleted below.
Step 1: Premigration data collection
The first step toward a successful data migration effort is to collect and correlate all host and array data necessary for the migration. The tasks involved here include:
identifying hosts for migration;
collecting migration host data (host audit);
collecting array data (array audit);
collating zoning and masking information; and
correlating all data.
Step 2: Determine remediation requirements
Assess what remediation requirements will need to be addressed prior to starting the data migration. This process reduces surprises and errors, while putting in place procedures to handle necessary remediations. The three tasks to consider here are:
conducting migration and destination remediation analysis;
doing host remediation; and
documenting source and target array software versions and compatibility, as well as fixing or remediating incompatible differences.
Step 3: Preparation stage
This step involves preparing hosts, initiator storage and target storage systems for migration. The 18 tasks involved here include:
configuring source and target array to facilitate migration;
creating relationship between source and target array;
collecting all host storage configurations, including installed OS, cluster status and disaster recovery partner;
collecting each array's configuration, including RAID, FA port and local/remote replication status;
collecting SAN fabric configuration, including directors, switches and gateways, and oversubscription details per port -- fan-in/out ratios;
planning storage layout at target arrays;
creating migration configuration and pairing scripts for source and target array;
creating FA port mapping and target array LUN assignment configuration scripts;
grouping migration hosts in cooperation with business units;
updating all operational backup and recovery scripts;
performing all host audits prior to migration;
creating and running R1 to R2 relationship pairing scripts;
creating and running target array mapping and masking scripts;
applying all hosts zoning configurations; and
confirming host connections at target array.
Step 4: Launch migration processes
Here are all the tasks involved with actually migrating data to the new array. They range from making sure data is backed up or otherwise protected to stopping applications and shutting down hosts to creating scripts for the new array relationship. Once the migration is complete, the tasks are reversed, bringing hosts back up and restarting apps. The specific tasks to follow include:
quiescing (stopping) apps, and then shutting down hosts;
creating and running scripts that split new Array 1 and Array 2 relationship once data has been synced;
bringing hosts back up to check and validate data;
restarting applications; and
getting business units to sign off that data migration is complete.
Step 5: Validation, reboot and cleanup
Validate the migration, bring up all apps with new target storage and disengage old storage. Look for, correct, clean up or minimally mitigate errors. Complete the following tasks:
checking for any activity on old arrays;
creating and running scripts to delete Array 1 and Array 2 relationship;
creating and running scripts to unmap devices from FAs on source array;
creating and running scripts to dissolve meta configurations on source arrays;
creating and running scripts to unmask source devices from migrated hosts;
rebooting hosts and confirming new target devices' visibility;
cleaning up zones and removing all unused zones;
checking final login table on old arrays; and
updating device groups if used.
Step 6: Final documentation
Complete, document and validate the migration with this final task:
Creating reports and updates documenting the entire process.
Manual migration vs. scripts
Planning, discipline, focus and attention to the details … make the difference between success and failure with storage data migration.
The majority of data migration projects are managed manually, usually using Microsoft Excel spreadsheets. This approach requires multidepartmental processes, procedures and, most importantly, discipline. This is because the data migration spreadsheet must be continuously updated by multiple administrators.
Scripts are a common workaround. The good thing about scripts is they can be customized for the specific environment, storage, servers, applications, vendors and infrastructure. The bad thing about them is they're rarely documented, tested for quality assurance, patched or updated as the environment changes. They also must be rewritten for every SAN data migration project. Scripts always look like a good idea until they're implemented.
Data migration technology timesavers
There are four technology workarounds to this manually intensive process, and while each has tradeoffs, all of them can significantly reduce data migration workloads and errors:
Use physical software-defined storage appliances or virtual SDS appliances (VSAs).
Make sure the new SAN is a newer version or model of the old SAN and replicate between them.
Use a scale-out SAN system.
Use VMware vSphere Storage vMotion.
It's expected that physical and virtual SDS appliances will simplify SAN data migration. They enable storage systems behind the SDS appliance or VSA to be migrated without touching any of the servers or infrastructure in front of the virtualization appliance or system. The tradeoff is they add considerable upfront and ongoing costs that increase the TCO. In addition, complexity is increased from having to manage all the SDS appliances or VSAs, hypervisor resources for the VSA and back-end storage arrays. And there can be additional complexity when going through a tech refresh for the SDS appliance -- although this is usually handled through sequential migration of active-active appliances or scale-out appliances. It isn't an issue for VSAs.
Staying with the same storage vendor and using the latest version of the SAN being migrated are a common, but not necessarily perfect SAN migration workaround. It requires a replication license on both systems, and all of the servers connected to the storage must be reconnected to the new storage. It's generally a solid workaround.
The use of a scale-out SAN system has grown in popularity. With most of these systems, processing and capacity can be upgraded transparently in the data center. This approach is typically as simple as adding a node and using the system software to migrate behind the scenes from the node to be retired to the new node. After the data is migrated, the old node is retired. Rinse and repeat until all nodes are upgraded. Connected applications, servers, virtual machines (VMs) and containers aren't disturbed with this approach. Erasure coding in many of these systems can eliminate the requirement to migrate the data at all. Simply add the new node, and retire the old. The data is quickly rebuilt on the new and remaining nodes.
VMware's Storage vMotion has become another popular workaround. It migrates the data from each of the physical SAN array LUNs assigned to that VMware server to LUNs on the new SAN array. The virtual LUNs assigned to the individual VMs don't change, masking the actual physical change in the storage arrays. The downside is that this only works for the VMs running in a VMware infrastructure. It also doesn't provide any SAN pathing remediation or VMware host remediation. Also, it doesn't move data from LUNs that aren't assigned to that particular VMware host, and there's no data migration testing or validation.
Of course, all four of these workarounds assume a commitment, or lock-in, to a single vendor. The first requires a commitment to the SDS vendor, the second to the storage array vendor, the third to the scale-out storage vendor and the fourth to VMware. There really is no such thing as a free lunch.
Storage data migration success
Storage data migration will continue to be a daunting effort. This is why many SMBs often turn to their storage vendor's data migration professional services. Yes, it's expensive, but it gives you a way to hold your storage vendor accountable. Yet, even this workaround has issues. It doesn't automate any of the processes but, instead, just outsources the responsibility.
Whatever the choice, it's planning, discipline, focus and attention to the details and tasks included in the data migration checklist outlined here that make the difference between success and failure with storage data migration.
Click the image to download
