What is the Agile test automation pyramid?

What is the testing pyramid in Agile?

The test automation or testing pyramid in Agile is a testing strategy that determines what types of tests should be included in an automated test suite for a given software product and in what sequence they should be performed.

The components of each layer can vary from one organization to another. However, the most commonly accepted structure has unit testing as the bottom layer, integration testing as the center layer and end-to-end (E2E) testing as the top layer.

The different levels of a test automation pyramid

The test automation pyramid has three different layers.

Layer 1 (bottom): Unit testing

Automating unit tests offers the best ROI to the organization, which is why most testing pyramids place them at the bottom.

Unit tests involve testing small units of code. They are the least expensive to write and maintain and are meant to validate individual components or functionalities. Many such components and functionalities get added as development progresses, so it's essential to run the unit test suite quickly multiple times during the initial stages of development.

Unit tests provide immediate feedback to developers to improve code efficiency and quality from the development stage. They help development teams build comprehensive, high-quality products during the software development lifecycle (SDLC) and form the largest part of the pyramid structure.

Layer 2 (center): Integration testing

Integration tests constitute the center layer of the pyramid. They test how various pieces of code interact with other pieces of code in the software. These disparate pieces can include application programming interfaces, web services and databases, among other components. Breaks here can affect the functionality and performance of the entire product.

Integration tests evaluate components in a group after those components have already been unit tested. Integration tests are not run as frequently as unit tests and are run only at key points during the SDLC. These tests help to confirm that the various components work together seamlessly and as one coherent entity before the software is released to users. Unlike unit tests, integration tests require a preproduction environment to run.

Layer 3 (top): E2E testing

E2E testing ensures that the entire software functions as it is meant to. When performing E2E tests, testers check the product's functionality and flow to confirm that it works correctly from start to finish.

They also try to imagine the end user's perspective. This enables testers to understand how a real user might use the product and then design the test accordingly. The test design also considers various possible user scenarios, which provides for more comprehensive E2E testing and more useful feedback to incorporate prior to product release.

As with integration tests, E2E tests often involve testing how the software communicates with external components. These tests involve numerous external dependencies to avoid any dependency issues and are meant to confirm that the entire application works correctly in an environment that closely mimics real-world production environments. For these reasons, E2E testing is resource-intensive, is time-consuming and can slow down the SDLC and product release. This is why experts recommend running E2E tests only at significant milestones and under real-world conditions.

Benefits of the testing pyramid

The test automation pyramid streamlines the testing process in Agile software development. The pyramid's hierarchical structure and clear logic provide a systematic way for teams to approach software testing and to improve speed, efficiency and efficacy.

By providing a clear progression of tests from the most accessible tests at the bottom to the most complex tests at the top, the pyramid enables teams to better plan and prioritize testing tasks without rushing or cutting corners. They can fix errors and bugs earlier and faster, reducing the need for costly fixes or do-overs later in the SDLC. This helps to control costs and also prevents time-to-market delays.

The pyramid works well in Agile environments since it promotes automated unit testing, integration testing and E2E testing. For example, development teams can use frameworks like Selenium to write test scripts that can automatically test applications to ensure that they function as expected. The automated testing process requires less human effort and offers feedback faster.

Agile teams are also encouraged to use practices like test-driven development. In TDD, the tests are written before the code. By adopting this approach, teams can better define their development goals and ensure that any changes to the code don't affect the product's existing functionality. The process of writing a failing test, writing code to make the test pass and modifying the code often results in reliable, bug-free code that then translates into a reliable, high-quality application.

Other benefits of the testing pyramid include the following:

• By following the logical structure, teams can avoid executing costly, brittle E2E tests more often than they are needed.
• This structure optimizes resource use. More resources can be allocated to perform numerous unit tests, and fewer resources are required to perform complex and pricey integration and E2E tests.
• Feedback becomes available quickly, enabling teams to fix bugs quickly and continuously improve code quality right from the get-go.
• The structured top-to-bottom approach ensures broad test coverage -- from individual features to integrations to the system as a whole.

Automated testing and the development of the Agile testing pyramid

In automated testing, software tools execute pre-scripted tests on a software application or program component to ensure its functionality. Automated testing makes it possible to run tests quickly and repeatedly, and the software involved can provide reports and compare results. Automation also minimizes or eliminates many common problems that result from manual testing, including the following:

• Missed deadlines.
• Code quality issues.
• Human error.

The order of testing can also impact testing success and the success of the Agile development project as a whole.

The pyramid was introduced by Mike Cohn in his book, Succeeding with Agile: Software Development Using Scrum. His idea was to encourage teams to approach automated testing at different levels and to allocate resources and efforts based on the complexity at each level.

Explore our comprehensive test automation guide for IT teams, which walks readers through benefits and challenges, how to craft an automated testing strategy and how to compare tools.