Guido Vrola - Fotolia
Dive into functional testing and non-functional testing approaches
Don't get trapped in a white or black box -- or the gray area in between. Here's why functional and non-functional app testing are complementary ways to ensure end user satisfaction.
There isn't a universal definition of functional testing and non-functional testing. The delineation really all depends on how you test.
Generally speaking, functional tests vet user interactions with software: their workflows and how the application responds. Functional tests include integration and regression tests as well -- everything that checks whether the app does what it is supposed to do. Non-functional tests put software through its paces outside of the user interaction. These scenarios push application performance and security, looking for problems that ultimately would affect the user.
The QA professional aims to provide a quality application that works across the board for its target users. To get there, you'll mix functional and non-functional tests. Let's break down both functional and non-functional testing by how they work, the kinds of software tests that each encompasses, and how they fit within a modern software testing approach.
Functional testing mimics a user
Functional testing, most experts agree, is a type of black-box testing. Testers provide input and analyze the output, without worrying about how the internal structure operates. Functional testing depends on requirements, acceptance criteria and use cases, which are how the developers define the application functionality that QA must test or measure. Testers seek to answer how well an application function meets its given requirements, and if the application does function as developers expected.
Functional testing evaluates how the customer, also called the end user, works in the application. Testers, for example, seek to find out what workflows end users perform and how they use an application. Functional testing is critical to provide a quality user experience.
Testers can perform automated or manual functional tests, as long as they appraise the application's functionality. For example, a banking application might apply a functional test that assesses this sequential customer workflow:
- log in, or authentication;
- select an account; and
- add, transfer and remove funds.
As a QA tester, I develop and execute test cases that exercise user authentication. To perform a functional test, I also select a checking, savings or investment account in the banking app. From there, I deposit money to each account type. Then, I transfer money between each type of account in varying denominations. Finally, I remove funds out of each account type. These are all actions a target user will take in the application, expecting the sums to be accurate each time.
During this functional test, the tester assesses how the application performs each of these actions and compares the assessment to the expected results, which requirements or acceptance criteria define.
Rather than test against functional requirements, testers could opt to test by how they expect the application to function. For example, I'd expect to see an account balance increase if I add funds, and decrease if I remove funds. While testers can apply basic logic to functional tests, it's more effective to test against a set of expected results, whether they are requirements, acceptance criteria or use cases.
Functional testing typically includes various types of tests, including:
- regression tests, which assess whether code changes adversely impact an app;
- integration tests that look for bugs in between combined units of software; and
- user acceptance tests, where end users execute a final check on the software.
The field also includes, at some level, security and performance testing, but this is where the demarcation of functional testing vs. non-functional testing tends to blur. Many experts consider security and performance testing to be non-functional testing. Yet, the box a QA professional puts each test in depends on her approach. If she undertakes a thorough authentication test, it likely includes a level of security testing and should catch performance problems. After all, the user expects correct sums after transactions, but also expects that money to transfer safely and the totals to update quickly. If functional testing on a feature takes more than 30 seconds to execute, for example, then the QA person should note a performance defect.
Ultimately, functional testing and non-functional testing coexist, but the depth of coverage varies.
Non-functional testing mimics live conditions
Non-functional testing assesses elements outside the application's customer workflows or functions. It's a way to appraise the structural elements of the system, such as the hardware or platform the application runs on, and how the app responds to various user loads. Unlike functional testing which has a black-box setup, non-functional testing is more like white- or gray-box testing.
Non-functional testing can evaluate the security across the application. Can an attacker hack a URL and gain access? Can malicious actors inject SQL into any fields?
Performance tests determine how well an application functions in potential real-world conditions, or extremes thereof. In the banking example, does the application hang if 5,000 users transfer funds at the same time?
Similarly, non-functional testing includes performance testing without regard to the customer workflow. QA testers will execute performance tests against each page, field or movement between pages of the application. They test how long it takes to move between pages and how quickly each page loads.
Usability testing, typically defined as a non-functional type, evaluates how intuitive application functions are to its target user. Can the user easily find the button he needs, and does the workflow make sense? Or, does he click Help or, worse, abandon the page?
Types of non-functional testing include:
- interoperability testing to check that software or system components can integrate with each other;
- Load testing, in which the app is pushed to the limits of its specifications;
- usability testing that exposes flaws in the software through user interaction; and
- endurance testing, which measures system performance when subjected to maximum loads over time.
In the day-to-day world of software releases, functional testing and non-functional testing overlap. When I test the functionality of an application, I include non-functional tests, even if it's on a small scale. Similarly, I won't do a full spectrum of security tests at this stage, because that requires a greater level of depth than what's appropriate for requirements validation, but I'll do some. This approach helps me keep the target user in focus. They want the app to perform as customers expect, easily, quickly and securely.
