There are many QA circumstances where the right call is to set up a staging environment: defects whose root causes are different between the test and production environments; multiple disruptive production outages affecting customers; a new application in development that requires a preview of the production environment for go-live authorization.
If any of these cases sound familiar, then it's time to design and test in a staging environment. And if you already do staging and issues persist in production, then it's likely you're missing some staging environment best practices that would optimize the setup.
Work in the staging environment is one of the most important parts of a QA strategy since staging is the last place to catch defects before production. More than in any other test phase, the staging environment's design is critical. Let's look at ways to build and use staging environments that ensure successful production releases.
How to mirror production
A good staging environment must mirror production in all respects. This includes not only in the codebase deployed in both places, but also the hosting setup, including network, server and tooling configurations; your team needs to reproduce production's databases and storage capabilities too. It's an ideal practice to equip the staging environment to handle as close to a production load as possible, although that means the organization must buy, license and manage duplicates of everything. Weigh the effort and expense against the amount of risk a staging environment mitigates.
A staging environment setup that's truly production-like can be especially complicated and time-consuming to run if that infrastructure is on-premises. For cloud-hosted applications, the task to spin up an additional production configuration is considerably easier. Virtualized staging environments do not require long-term maintenance; you create and dismantle them as needed.
Another staging environment setup best practice relates to the data available for use -- it must resemble data in production as closely as possible without violating privacy regulations. If data replication isn't possible for staging, set up sufficient quantities of data in the states that occur in production to enable realistic test scenarios. For an application that is not yet in production, create test data based on user personas and their user value stories to anticipate customer input.
To ensure that the staging environment does not become contaminated with buggy code, use tools that facilitate rolling back the last deployment to the environment. This consideration is particularly important for staging environments that are the final phase in a CI/CD pipeline.
Test effectively in the staging environment
If you only subject code to a cursory inspection in staging, the environment becomes a rubber stamp with no added value.
The other half of staging environment best practices relates to how it gets used. A QA team can use a good staging environment for all production deploys. But, if you only subject code to a cursory inspection in staging, the environment becomes a rubber stamp with no added value.
Test all of the code in staging, and run all appropriate tests. Start with a smoke test to verify that the team deployed the code correctly. Then, execute user acceptance testing to simulate real interactions in the simulated production environment. Since user acceptance testing is usually part of an overall test strategy, executing it in a staging environment doesn't impede release velocity.
If a production issue arises that is so severe that it requires an immediate fix, run a sanity test in staging before the update goes out. This approach is the best practice no matter what development methodology your organization adheres to.
While you should consider and test the UX throughout the development lifecycle, the staging environment is a great place to catch UX issues that your team missed in earlier testing. Issues with colors, location of radio buttons and navigation might only become visible in an environment that mirrors production.
Testing in production vs. testing in staging
Production-level QA, with techniques such as A/B testing, canary releases and fault tolerance testing, has grown, especially for Agile and DevOps shops. However, these "shift-right" techniques do not replace staging tests or eliminate the need for a staging environment. For example, you need a safe place to do destructive testing as well as full performance and load testing.
Use testing in production to complement what you do in staging. Production changes yield valuable information about customers, including who they are, what types of features they value and what their pain points are. You can use this information to develop or expand on personas and user value stories, and generate acceptance test cases to execute in the staging environment.
Dig Deeper on Software testing tools and techniques
