Getty Images
What regression testing types make sense for your applications
Craft an in-depth regression testing plan that uses multiple types to test all facets of an application's functionality and ultimately lead to cleaner development.
When development teams create a plan for regression testing, they need to weigh how quickly the tests can complete against how the tests cover an application's full functionality. It calls for a QA team to craft a regression test plan that generates the most value in the shortest amount of time. The more test coverage regression testing provides, the better the result for your customers.
Regression testing occurs at the end of a sprint in Agile or Agile-like software development teams, if it occurs at all. Typical regression tests last one to three days maximum, and teams will need to cover all the possible functional workflows.
Regression testing also has to account for both mobile and web users. Factors that make regression tests time-intensive include the complexity of the code, application functions and back-end processing. Similarly, variables such as team size and how much high-value test coverage is required will determine the preferred style of regression testing.
Let's explore which full system regression testing types are well-suited for Agile teams -- with, say, three to six QA professionals -- and how these tests deliver value.
Repeat execution across test systems
A popular Agile regression test type is retesting stories and bugs in a staging or pre-production release server. This method ensures code is packaged and deployed from the original development server to the staging or secondary test server.
Teams can also add smoke tests to cover the main workflows within the application for further test coverage. Additionally, if the regression testing simultaneously applies to a mobile and web-based app, there needs to be retesting for both code builds.
A downside of this method is how much functionality isn't touched by these tests. The retest regression method includes a general smoke test and the retesting of bugs and stories in a second server. However, this method lacks an end-to-end or system test, integration testing involving back-end processing or an assessment of customer workflows. When these tests aren't performed, it can create significant gaps in test coverage across the application.
It's one thing when teams confirm code can be successfully packaged and deployed. But when teams simply retest stories and bugs on a non-development server, that's not regression testing. It proves if the deployment succeeded, which isn't the answer you're looking for from these questions.
A better way to prove if the new code properly deploys is to create and execute a series of integrated unit tests when the code is checked in or built into a package.
Workflow or end-to-end system test
It's crucial to cover a lot of functionality quickly in Agile development, like all the functions, customer workflows and back-end processes. If teams want to reach this speed and coverage, they won't need to create a separate regression test suite. Instead, they should create workflow tests.
The easiest and simplest way to create these types of tests is using exploratory test stories as tours of the software application. James A. Whittaker's book, Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design, remains a good resource on the approach.
Tour-type regression test suites cover more application functionality, largely because testers write these tests from the customer perspective while keeping the user workflows in mind. When these tests are written well, there may be seven or fewer tests that cover the full application. Additionally, this style of regression test provides built-in, back-end integration and performance testing. QA professionals often skip integration and performance during regression testing because of the extra bandwidth required to perform them.
Regression test authors need to know certain elements of the application -- such as their integration points, database impacts and all the back-end systems like APIs or messaging data connections -- to write these tests. A good candidate may be a QA professional with a broad range and depth of experience in the applications. After someone writes the tests, have the rest of the QA team review them. Accordingly, all team members will have input in case the tests miss or misrepresent any functionality. As a QA team, plan to execute the tour tests during each regression execution cycle and make edits or changes as needed.
If teams want to develop more well-rounded members, spread the responsibility of regression test amongst the team, and switch each cycle to build more familiarity with the different roles. Tour tests are written in a casual, story-like format and can be easily transferred for other teams to use. Tour tests can also be shared with customers, as well as user acceptance testers. Finally, a full suite of tour regression tests provides full coverage documentation to use for training new resources.
Teams can increase their overall test coverage by creating workflow or system tests as exploratory tour tests. Additionally, such tests are easy to share with other teams or customers and provide overall system documentation.
The only disadvantage of tour regression tests is that they're harder to label as covering a specific function because they tend to cover multiple functions. However, teams can overcome this disadvantage by noting the functionality covered in your test strategy documentation or as an addendum to the test itself.
The best way to release a high-quality application is to routinely test the customer workflows across the application system, including back-end integrated processes and performance. All these steps are critical to your software application and the customers using it.
