your123 - stock.adobe.com
Here's why you still need exploratory testing documentation
While documentation is an important part of exploratory testing, it doesn't have to be extensive -- and it shouldn't get in the way of learning about the software.
While exploratory testing is, by definition, unscripted, that doesn't mean it should be undocumented.
Exploratory testing documentation differs from its counterpart, scripted testing, in several ways -- including the point during the test cycle at which it occurs. In addition, documentation for exploratory testing differs in terms of detail, format and general best practices.
Understand exploratory testing
To understand exploratory testing documentation, you must first understand exploratory testing itself.
In their research paper, "Exploring Exploratory Testing," Cem Kaner and Andy Tinkham define exploratory testing as "any testing to the extent that the tester actively controls the design of the tests as those tests are performed and uses information gained while testing to design new and better tests."
This includes test design, which usually involves documentation.
However, exploratory testing isn't a type of testing; testing types are used to meet specific goals. For example, security testing finds vulnerabilities in software. Exploratory testing also isn't a test technique. Test techniques flush out specific types of defects. For example, negative testing techniques ensure software doesn't do something it is not supposed to do.
Instead, exploratory testing is an approach to testing; it's a way for development teams to carry out a test. A test approach should account for the overall goal of testing. If the goal, for instance, is to verify that software functions in accordance with acceptance criteria, the usual approach is to use scripted test cases traced directly to user stories. However, this approach doesn't necessarily include all the edge cases.
If teams want full test coverage, they should combine test approaches and include exploratory testing to find defects that might be missed by testing only to the software requirements.
Since exploratory testing is often part of a comprehensive test plan, some documentation is needed.
Scripted vs. exploratory testing documentation
The three main differences between documentation for exploratory testing and scripted testing are when it occurs, how it's designed and how it's captured.
1. When documentation occurs
In traditional scripted testing, teams complete documentation prior to test execution. Test cases are designed, written and then executed. They may be used to develop automated test scripts and reused in regression testing.
With exploratory testing, test design happens simultaneously with test execution. While some components of exploratory testing -- such as charters and mind maps -- involve planning prior to execution, most of the documentation takes place during execution.
2. How tests are designed
Test design is one of the most important documentation activities in both exploratory and scripted testing, but it's accomplished quite differently. In scripted testing, developers design test cases in a specific format, detailing each step and its expected result. In exploratory testing, test design involves providing parameters, but specific steps and flow aren't defined. This enables the tester to control the direction of test execution more effectively.
3. How tests are documented
In scripted testing, teams usually document test cases using a test lifecycle management tool or in an Excel sheet or Word document. Test results and any defects are also documented in these tools. In certain scenarios, such as regulated testing projects, teams need to collect test evidence and include it in the documentation.
In exploratory testing, tests are documented using mind maps, decision tables, flow charts and even notes. In addition, some test management tools -- such as Test IO, Testpad and others -- have exploratory testing features that enable testers to record their actions through the system. Codeless test automation tools can be used to record actions and, thereby, create documentation as well.
Guidelines for exploratory testing documentation
One of the best approaches to document exploratory testing is to start with a session-based test charter. Then, use a record and playback tool to capture the actions and results.
A session-based charter describes the mission or the test scenario, gives generalized instruction on how to test and notes any issues that arise. It is usually timeboxed. The record and playback tool then records the exploration of the charter. Teams still need to document the defects, obstacles and learnings.
Mind maps are another format that can be useful for exploratory testing documentation. Although they can be used during test execution, it's easier to plan them out before the test starts. With this approach, the tester documents all the predetermined high-level scenarios, but they continue to document throughout the test.
The main advantage of these tools is that the results are reviewable and repeatable. Repeatability is especially important for defect management and to plan a test suite for regression testing.
However, find the right balance between repeatability -- especially when defects are found -- and freedom of execution. One of the goals of exploratory testing is to learn about the application. As such, documentation shouldn't interfere with the learning and interrogation processes.
