How software testing affects a successful multi-cloud strategy

Where multi-cloud makes things complicated

Four specific areas where development and deployment complexity can increase with multi-cloud are:

• Implementation across multiple clouds. While common features may be available across all clouds, their implementation usually varies at least slightly from one to the other. So, different software versions will be required if developers must prepare a single component for deployment in multiple clouds.
• Widely spread-out components. Application performance and quality of experience vary depending on how the components in a workflow are spread across the clouds, making performance monitoring and assurance difficult.
• Tuning deployments for each cloud. The same operations tools may not be available in all clouds. Likewise, developers may not use certain tools in the same way within each cloud. As a result, team members may have to tune deployments and redeployments differently in each cloud. Additionally, redeploying from one cloud to another may have to be validated for every possible cloud combination.
• Ensuring connectivity. Connectivity between internet users and the various public clouds, and between the public clouds and the on-premises data center, will vary. Accordingly, network addressing, and traffic management practices, will have to accommodate the differences.

Where to address multi-cloud risks

Any increase in complexity creates a risk for problems in application performance or availability, security and compliance and/or cloud cost overrun. These risks stem from two factors. The first is that multi-cloud may require deployment of the same logic, and developers may need to code the logic in different ways to accommodate cloud API differences, which multiplies the number of application pipelines. The second is the risk that workflows between the clouds in this setup will vary in ways not anticipated or accommodated in the applications' design. It's this variability that multi-cloud testing strategy must address.

Team members can address application-multiplication risk during the development and unit testing cycles. However, this sort of multi-cloud testing requires the clear separation of software components that need per-cloud customization. This separation ensures code never crosses over to a cloud where services and features aren't compatible. It also means that the development and testing process -- up to the point of load testing -- should treat each cloud-specific variant as a separate application or project, with its own development pipeline.

Through such a setup, the relationship between each cloud variant and the baseline logic for components must be maintained so that parallel maintenance is efficient. Some organizations will maintain a base code set for each component, and then branch off it to cloud-specific versions to ensure that logic changes will apply to all cloud versions in a coordinated fashion. When a logic change to a software component that previously didn't require cloud-specific code creates a need to customize for one or more clouds, developers then fork the single base version for each cloud that needs accommodating. Documentation of all of this is critical.

Where workflows become a factor

If the development team partitions cloud-specific code using per-cloud pipelines, development and testing practices are the same as they would be for a single or hybrid cloud. However, multi-cloud requires special practices and even tools for load testing. After all, load testing must simulate the conditions that create variations in workflow between the clouds. Changes in workflow will expose new logic, which QA professionals must test.

There are two reasons why workflows would vary. First, there may be features or capabilities best for one specific cloud provider to handle. And this will mean that work must flow to and from that provider when these features/capabilities are required. Second, an application may selectively use one cloud to back up another or to support scaling under load. In the first of these cases, it's important to focus on test data input that exercises this feature/capability-specific logic. In the second, it's necessary to create the availability or workload conditions that change multi-cloud workflows and component placement to ensure that the logic works and QoE doesn't become unacceptable.

It's not necessary to use special test data generation tools for multi-cloud testing. But if QA elects to use them, it's important to use them properly. Proper usage starts by ensuring that you provide test data injection concurrently across all the clouds where user access is supported. Such access is always the case where a multi-cloud decision was made to accommodate a distributed user base. Also, be sure that the test data injection for each cloud is local to that cloud, meaning generated in the same geographic vicinity where users will connect.

The other critical requirement is to emphasize the workflow-centric monitoring of the test, and in turn, extend that same view to problems unearthed during this monitoring. You're looking for situations with unique multi-cloud risks, and the subsequent responses to specific workflows for scaling, failover, or when the application invokes access to cloud-specific logic. These special situations will have to be triggered by test data and then measured specifically through the range of components and across all cloud boundaries.

Random test data alone will not always activate the special situations you're trying to test, so coordinated test sequences that are synchronized across all the clouds where users connect are essential. When you find something, follow the symptoms through the entire workflow to ensure you've addressed the problem fully.