soak testing

What is soak testing?

Soak testing is a type of performance testing that gauges how an application handles a growing number of users or increasingly taxing tasks over an extended period of time. Soak testing is both a type of nonfunctional test and a form of stress testing. The process informs software testers whether an application performs satisfactorily after hours -- or days -- of usage. Soak testing uncovers software issues that only become apparent after it runs for extended periods of time, such as a gradual memory leak.

During soak testing, a quality assurance (QA) team records metrics for key performance indicators, such as throughput, response time and memory usage, and compares them from when the test begins to when it ends.

A QA professional incorporates soak testing before an application or an app update reaches production, where it is available to real users. Soak testing typically takes place alongside system testing, which examines software components as one system to make sure they align with specified requirements. Other nonfunctional tests often occur at this test level, such as load testing.

A testing team could choose to run soak tests overnight or on a weekend to avoid disrupting other members of the software development team during normal working hours.

Soak testing vs. load testing vs. stress testing

Soak testing can be confused or conflated with other types of performance tests, including load and stress tests.

Load testing subjects an application or system to a level of work it is expected to handle. This process aims to determine the ideal limits of work that an application or software can handle. Stress tests expose applications to more than 100% of the expected level of traffic -- aka the intended load -- to determine the threshold at which the software or system can no longer reliably perform. Unlike load testing, stress testing is typically performed until the target breaks.

Soak testing differs from both methods as it focuses on long-term performance sustainability -- not determining the upper limits of a system's performance capabilities. A soak test ascertains if an application functions when it's subjected to as much as 100% of its intended load for a prolonged period. In a soak test, testers monitor the effects of accrued load.

Soak testing tools

Many tools can execute soak tests in accordance with other techniques. Typically, a QA professional uses a load testing tool to execute a soak test and program in periods of high, medium or low traffic as needed, as well as the duration of the test. If the tool has a maximum duration, the tester can use the results and extrapolate them to estimate performance over longer time spans, such as a week or month.

Examples of tools and automated systems with soak testing capabilities include Apache JMeter, LoadUI, WebLOAD, Appvance IQ and a variety of plugins on GitHub and Jenkins.