Guide to creating a cloud migration testing strategy
Follow these guidelines to help craft a strategy for cloud migration testing, from key tests to run to common challenges and best practices -- and why everything involves security.
Moving an application to the cloud can be a unique challenge. Users don't directly interact with servers or other underlying components of on-premises IT infrastructure, but in the cloud everyone sees and works with an application. As a result, application testing is a critical part of any cloud migration.
A formalized framework can help IT teams build out a cloud migration testing strategy, and ensure that an app is working as it should in the cloud. Let's go through some of the main areas to focus on, best practices to follow and problems to anticipate and resolve.
Why is cloud migration testing important?
Cloud migration testing helps IT teams ensure the app continues to perform as it should after it moves to the cloud, and also ensure a better UX. To do this, they must gauge the app's performance on both sides of the equation -- how it ran on premises, and how it works once it's in the cloud.
Metrics from cloud migration tests help identify and quantify any issues -- otherwise, comparisons based on guesses drain IT staff time and resources. Key metrics include application start times and response times, performance during peak demand and off hours, and usability on various platforms (desktops, remote connections and mobile devices).
This article is part of
What is cloud migration? Essential guide to moving to the cloud
Also, cloud vendors may issue updates that could cause performance issues to your app. Validate this through your testing metrics, and work with your cloud provider to find out what happened and what adjustments will correct those issues.
Finally, cloud migration testing reveals where IT teams can adjust performance or UX to justify keeping that application in the cloud.
How cloud migration testing differs from traditional app testing
The main difference in testing applications on premises versus apps that move to the cloud is that you have to accommodate the cloud's scalability, and additional integrations and dependencies. Some of those integrations might be difficult to identify and understand. Your cloud testing framework may differ from a framework to test something that is on premises, and some tools might be different, such as for load testing or pen testing.
Keep in mind, though, that the actual end goals for testing should not change from on premises to cloud. Different goals or guides mean you're no longer testing apples to apples. Use many of the same steps and tools to provide a clear picture of your efforts to migrate your application to the cloud -- changes to the testing framework or methodologies may skew results.
Types of testing for a cloud migration
The goal with cloud migration testing isn't to test every possible feature and function, but you need a solid cross section of tests and metrics to ensure your application performs as expected. Several different kinds of tests are especially important for a cloud migration:
- Functional validation. Confirms that the build meets requirements for component and service functionality, on both sides of the cloud migration effort. Anything missing tends to jump out quickly.
- Performance. This spans several metrics to measure how an application performs during real-world conditions: ability to handle data volumes, capacity loads, and CPU and memory usage.
- Integration. Your application likely ties into other services and apps to share data, in the cloud and on premises (and possibly both). IT teams must verify these connections still work in the cloud. This can be tricky if dependent services are in a different cloud or remain on premises.
Other aspects to test: If your application has a dedicated portal or requires interface tweaks to run on mobile devices, test it on both Android and Apple platforms. That's as true for a cloud-migrated app as for one that runs on premises. Testing every possible device type may not be feasible, so work with your user base to determine the best mobile platform representation.
Also, test on both cellular networks and Wi-Fi networks, because different data speeds impact the app's behavior. Do this outside of your company's Wi-Fi network -- the internal side of your network might access services differently than an external Wi-Fi network, which could cause the application to function differently or not at all.
Finally, test for printability -- yes people still do print, and for some it's a critical job function. Don't underestimate this potentially challenging technical problem. Printing from a cloud-based application to a local printer encounters security and network challenges that you don't have on premises.
Security considerations for cloud migration testing
A proper application security audit and adjustments to end-user privileges should happen before you begin to move an application from on premises to the cloud. The last thing you want is to introduce additional changes that muddle or complicate your understanding of how an app works and its interdependencies.
Your cloud migration testing should include a security validation that confirms three key factors:
- Who has access to the app? Start with the accounts and the features/functions that users would access. Adding more variables won't help or speed up the testing, so as a lateral move end users' access should mirror what they had with the app on premises, no more and no less.
- How have you ensured least possible privilege? As your data moves between different services in the cloud, what steps will ensure least possible privilege? Cloud vendors offer services and reports to help you with this; take the time to learn and use them.
- Will security changes impact the app? Always rerun application testing after you make a security change, as these changes can potentially affect performance. For example, will data encryption at flight and at rest change application response time, or add overhead to CPU cycles? This can be a subset of full testing, but you must revalidate at a competent level that the application still works as baselined and that your security change won't take the application offline.
7 common challenges in cloud migration testing -- and best practices to address them
Application testing is a challenge no matter how smoothly things go. Despite all your planning and testing, you will encounter issues. Nothing is ever as smooth as you hope, but most problems are solvable.
Challenge #1: How to prioritize app testing metrics
An application's UX and security are important to the overall application, but ultimately the application must work as intended. An application that lacks core functionality, even if it's absolutely secure or wonderfully streamlined for users, does no one any good.
Best practice: Balance design and security with critical functionality. Take the time and effort to address all factors -- functionality, UX and security -- early in the cloud migration process. These are not separate aspects that you can work on at different stages; they are interconnected, and you must treat them as such.
Challenge #2: Don't cut corners just to make the app run
It's critical to ensure the application works properly in the cloud, but don't adopt bad habits to achieve short-term goals. IT teams may reduce or remove security controls to get an app functional in a new environment and forget to restore those safeguards afterward. That may save time initially, but creates more and complicated work later and makes testing overall more difficult.
Best practice: Plan security from the start. Security permissions issues often arise and can impede an app's functionality, but resist the knee-jerk reaction to make large scale changes to get things running right away. Build time into your cloud migration plan to address these problems. (See the sidebar for more advice about cloud migration security throughout the entire cloud migration process.)
Challenge #3: IT alone can't do cloud migration testing
Cloud migrations are complex efforts that require many disciplines to ensure a migrated app works at scale and securely. IT has a major responsibility and can help lead these efforts -- but you can't just forklift and focus on the technical side without development's expertise.
Best practice: Form a diverse testing team. Form a cross functional team to provide app support and identify possible issues. The team must be nimble, and you need content experts to ensure all aspects are being looked at. Parking lot certain items in meetings so you don't overburden everyone with too much information -- some may get bored or tune out technobabble that's outside their expertise.
Challenge #4: Resist "easy" updates and scope creep
Enabling encryption for an app that's migrating to the cloud may seem ideal, and deceptively simple. But beware: Any change can unknowingly impact many other aspects of the app's operation, and increase both technical concerns and costs. This scope or feature creep in your migration testing can quickly expand to unsustainable levels.
Best practice: Test and vet additional functions. First, try to stick to the original plan for the application's core functionality -- the testing phase isn't the time or place for feature expansion. If you must add any minor new features or functions, get everyone's buy-in and test and vet those changes to understand their broader impacts.
Challenge #5: Balance testing cadence against competing priorities
Ideally you should perform continual security testing and at least weekly testing to validate data and handle error logs. Unfortunately, money, time and shifting priorities of IT and management can overrule those plans. The cloud costs money for every second it's in use. Staff requires time and money, and after an app's release most ops and dev teams move on to other projects.
Best practice: Plan around major updates, and push for more. IT always finds ways to fit what is required into what's available. Break down cloud migration testing into what management wants, needs and expects. Set up a testing framework for daily and weekly checks around major changes to the base application or platform. Push for more regularity if more resources are available.
Challenge #6: UX testing is narrow
Some IT teams rely on a regular pool of users for on-premises testing, or a few who are experts in the specific software. However, when you move your application to the cloud, you need a more comprehensive picture of your application. That means gathering data about the overall UX from multiple perspectives, and not just technical ones.
Best practice: Expand your user test teams. Coordinate user testing with a fairly wide pool of testers from different experiences and roles. Include people who are cloud-savvy and novice. Most end users don't know or care about the cloud's inner machinery, but they see your application and have certain performance expectations. And sometimes they might know it a little better than you do.
Challenge #7: Testing metrics overlook cloud cost concerns
In an on-premises migration typically you own all the pieces the app touches. In a cloud migration you'll incur costs for every process step, service called and data transferred to a cache or back on premises. There are integrations and dependencies to other cloud services, some of which may not be obvious. You may even use new cloud-based testing tools. When the bill comes due it might come as a shock and force you to adjust the app or even reevaluate the cloud migration.
Best practice: Regular testing reveals cost trends. Cost management isn't a central concern of cloud migration testing, but testing reveals exactly what an app does when it runs in the cloud. Thus, frequent cloud migration testing can help an organization get a better feel for data transfers, usage patterns and key dependencies. That means no surprises when the bill arrives.
