regression testing
Regression testing is a type of software test that assesses if changes to an application, or other related software components, introduce defects. A quality assurance (QA) engineer performs these exercises to see if modifications to code break or hinder the way in which the application works or how it consumes resources.
An application change or addition can cause unintended side effects, called regressions, and might even pop up in components or systems separate from the altered code due to dependencies. A regression test flags these adverse effects. An application change or code modification that necessitates regression tests can include:
- The introduction of a feature or functionality.
- A fix for a defect.
- Refactoring to boost performance.
- Alterations to an application's hosting environment.
A QA engineer executes a regression test at any of four test levels:
- Unit testing - a software development process in which the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation.
- Integration testing - a software development process which program units are combined and tested as groups in multiple ways.
- System testing - the quality assurance (QA) team evaluates how the various components of an application interact together in the full, integrated system or application.
- Acceptance testing - determines to what degree an application meets end users' approval. Depending on the organization, acceptance testing might take the form of beta testing, application testing, field testing or end-user testing.
Typically, a regression test occurs late in each test cycle, close to release; that said, don't discount the importance of unit regression testing, which narrowly addresses one unit of code at a time and reruns unit tests.
Regression testing must keep up with the speed at which an organization delivers software. Regression testing can be time-consuming and repetitive, requiring hours or days to complete. For this reason, a software team might choose to automate tests, both to reduce test execution time and free up the workers for other tasks.
Regression test techniques
Test case management helps a software development team determine whether a regression test is effective. A team assembles or orders its collection of test cases into a test suite.
One approach to regression testing is the retest all -- not to be confused with retesting -- technique, which, as the name suggests, entails the execution of every regression test case the team has written. While this method is thorough, it might be overkill for smaller releases. There are contexts that call for the retest-all technique, including when software is adapted for a new platform, language or culture, and when an operating system receives a major update.
If a software development team opts to build a tailored test suite for each test cycle, they can choose test cases according to their place in a priority system. A testing team might define a high-value test case as, for example, one with a high failure rate, or one that pertains to end-to-end or customer-facing aspects of the software. A test case that relates to added functionality, or checks critical application features, might also receive high priority.
A software development team can also follow a partial regression strategy, where they run just select test cases, but include tests related to any big and important features. This sort of approach is referred to as regression test selection (RTS) by some. For those who see a distinction between the two terms, with RTS a team only executes test cases that might be affected by -- or associated with -- the most recent changes to the code; with partial regression, the test suite also contains those tests which pertain to critical features.
To determine how widespread of an effect a change or build has on the software, the team can perform an impact analysis. Accordingly, an impact analysis informs the collection of test cases for the subsequent test cycle.
A smoke test, which typically precedes a regression test, only determines if the basic functionality of an application works. Regression test cases examine the software more exhaustively.
Whenever a test elicits a previously undiscovered defect, a tester should write a test case that similarly induces the bug.
Regression testing tools
An IT team can use a tool or platform to execute and optimize regression tests. A test management tool helps testers apply script automation, as well as trigger smoke and regression tests. In most cases, a regression testing tool leaves some level of manual test work for QA.
The regression testing tool market includes options with support for different programming languages and CI servers -- and by extension the CI/CD process. The market also ranges from low-cost or free open source options to feature-heavy commercial tools.
Open source regression testing options include Watir, Sahi and Selenium.
SmartBear's TestComplete and Ranorex Studio, two commercial options, test web, mobile and desktop applications. TestComplete is a front-end and functional testing platform, while Ranorex Studio is a GUI test automation framework. Each product integrates with a variety of CI systems. Functionize uses machine learning as part of its test automation capabilities and also integrates with CI/CD tools.
