Positive vs. negative testing: Differences and examples
Take an in-depth look at positive and negative testing. Learn how to use both to form the basis of a thorough testing approach through a series of real-world examples.
When testers approach a new project, they immediately think about testing the main user workflow or happy path. This is known as positive testing. However, it is critical to consider the alternative paths, error paths and failure testing, otherwise known as negative testing.
Positive and negative tests are important parts of both functional and non-functional testing and occur throughout the software test pyramid. Teams should use these techniques as part of both scripted and exploratory testing. Together, positive and negative testing form the basis of a rigorous test approach. Testers should always remember that there should be at least two tests per requirement, one positive and one negative.
Comparing positive vs. negative testing
Positive and negative testing are two of the most important techniques that testers can employ in their pursuit of quality.
Positive testing verifies that the software under test performs as the requirements specify. It ensures that the functionality does what it is expected to do. Positive test cases verify that when the function under test is exercised using valid input, the input is accepted and the user is able to proceed with the flow.
Negative testing, in opposition to positive testing, uses unexpected conditions to ensure the functionality does not do something that it should not do. Negative test cases verify that when using invalid input, the system does not allow the user to proceed with the flow as if the input were valid.
Test teams employ positive and negative testing by designing test cases or by including these approaches in exploratory test charters. The test cases for both positive and negative testing are developed with expected results. Test cases will pass if the actual results match the expected results. The difference between positive and negative test cases is the inputs. Positive test cases exercise valid inputs whereas negative test cases exercise invalid inputs or unexpected conditions.
Although both positive and negative testing are critical, when negative testing is not included in test designs or missed completely, it can lead to high-severity defects going into production. There are many cases in which the root cause of these defects can be attributed to missed negative tests.
When to use positive and negative testing
Both positive and negative testing should begin early in the software development process to catch defects as quickly as possible. Test teams should think about not only positive happy paths but also the negative error paths as they review requirements. For example, a tester might point out that acceptance criteria need to be added or amended to include appropriate error messages. As much as possible, developers should be encouraged to include both types of testing in their unit and component tests.
Teams should consider both positive and negative testing during test design, especially in health and safety critical domains. In any domain where development and testing are regulated and put through design control, both positive and negative testing might be required. This is because the types and the level of rigor of testing is determined by functional risk.
As much as both positive and negative testing are equally important and should be included throughout the development process, executing both is time-consuming, expensive and not always necessary. There are levels of test in which only positive testing is needed, such as smoke testing and sanity testing.
The smoke test is a high-level test that serves as a quality gate. It ensures that a new build's functionality is stable enough to execute the more in-depth test runs. Sanity tests are informal, unscripted tests that provide high-level verification that a component or module is working as expected.
Since negative tests can be destructive (e.g., creating errors and forcing the software to fail), they are not used in post-production deployment checkouts or in production environments.
Although teams don't typically use negative testing during smoke and sanity testing, it plays a critical role in non-functional testing and user experience. Negative testing is an important part of user experience testing. Testers exercise error conditions and use these to assess how gracefully the system handles erroneous input. For example, the tests might determine if an error message is generated and if the message provides the user with the information needed to correct the error.
Examples of positive and negative testing
Consider the following examples to contextualize negative and positive testing.
The difference between positive and negative test cases is the inputs. Positive test cases exercise valid inputs whereas negative test cases exercise invalid inputs or unexpected conditions.
User interface testing
Teams frequently use positive and negative test cases together when testing individual fields in user interfaces. First and last name fields usually require alpha characters. Therefore, the positive test case would test names with alpha characters, with those characters accepted as the expected result. The negative test case would exercise invalid inputs such as numbers and special characters. It seems simple, but what about names that include apostrophes? This is an example of where negative test designs can flesh out additional user acceptance criteria and ensure the application does not accept input that it should not.
Boundary value analysis
Positive and negative tests can be combined with the boundary value analysis technique to ensure complete test coverage. Testing teams use boundary value analysis to test ranges of input. For example, consider an application that should accept an age between 18 and 65. Positive test inputs would include 18, 19, 64 and 65; the expected result is that all of these are accepted. Negative test inputs would be 17 and 66 with the expectation that these inputs generate an error.
Non-functional testing
It is critical to consider both positive and negative testing in non-functional testing as well. One example of this is load and stress testing. Load testing verifies that the system can handle simultaneously the number of users specified in the requirements. This is a positive test. In stress testing, the negative test pushes the number of users above the expected limit to determine at what point the system will crash. SQL injection, which injects destructive code into a web application for the purpose of exposing security vulnerabilities, is an example of using negative testing during a security test.
Gerie Owen is a lead quality engineer at ZS. She is a conference presenter and author on technology and testing topics, and a certified Scrum master.
