Four critical continuous testing components to implement
Rapid continuous delivery is unlikely to go off without a hitch, unless the team relies on continuous testing. Expert Justin Rohrman explains how to get it underway.
Continuous delivery will fail unless you have a process to test software at the pace of development.
One of the biggest tangible changes in software development over the years is the pace of delivery. Development organizations went from delivering twice a year to monthly. A growing number of companies deliver new software to production several times every day. To enable rapid delivery, developers make more focused and smaller code changes, while creating products that can be built piece by piece rather than in a monolith.
However, in order to keep up with this continuous software delivery process, organizations also need continuous testing.
What is continuous testing?
Let's start with what continuous testing is not. In an average two-week test cycle, the testers join the process when they receive the build code from developers. They first wait through a sprint kickoff meeting and about a week of development work. There is a 50% chance that developers will write unit tests for their code. When testers receive a new build that can be deployed, it has a batch of code in a questionable state, and they discover bugs nearly immediately.
Continuous testing delivers information related to product quality, from the idea of a feature all the way through production. The best way to get a project done is by asking, "How can we test this?" There is a large skill, technology and process gap between testing in the last days of a sprint and driving the entire sprint with testing. How do we get there?
What pieces do you need?
Continuous testing relies on a set of processes, technology approaches and tools that make the feedback loop incrementally faster. Each individual piece improves that loop, but combining them makes a drastic difference.
Continuous integration (CI). The first step toward continuous testing is to get a CI system in place. At a base level, a CI tool takes code committed into a source code repository and builds it into the latest version of your product. In addition to producing a built package, CI systems also integrate with tools that run tests and produce reports on code complexity and test coverage. More advanced developer groups use CI in pipelines. Each development team has its own branch of the product. They commit to that branch, run a build and get test results on the whole product at any time without disturbing the work of other teams.
Environment provisioning. Environment creation always feels slow, even when it is not that bad. You have to grab the latest product build, move it to the server, run an upgrade procedure on the product and maybe the database, restart the services and hope everything starts up correctly. The 10 minutes it takes to perform that task becomes a burden when you repeat it 10 times every day. A container platform, such as Docker, makes provisioning take seconds rather than minutes. Container-based test environments include minimal OS components, so they are lightweight on IT resources when they spin up. Container image creation is a time investment, but these images are then ready for each new test environment. Consider managing the container-based provisioning step through the CI system as well.
Automation. CI and environment creation enable you to look at new changes as quickly as possible. However, that doesn't do much good if each new build is either dead on arrival or full of problems that can be found in seconds. You can improve first-time quality by testing a product with code. For example, a team adding a configurable drop list to an existing page in the product could write unit level tests of the new element and configured options. At the service layer, developers can go through permutations of data. Once there is a build, they can focus on all of the more complicated and surprising things users do.
Collaboration. Collaboration -- sometimes through developer pairing -- is the most important, and most difficult, part of continuous testing. Two developers work together -- or, even better, two developers and one test specialist -- and approach something called one-piece flow. The group can work on one change, uninterrupted, until it is ready to ship to production. The testers start with a test, and then, they write product code to make the test pass. After the test passes, they can refactor code to make it more readable or better-performing, or to satisfy the given code standard linters in use. The test specialist can make small code changes, shape the test strategy and ask what if-type questions. When the group is ready to merge into the main branch, the change is ready to push to production.
The processes delineated here seem like a steep hill to climb at once. But continuous testing isn't an end state; it is a pursuit. If you need a place to start, optimize your CI system to produce builds in a pipeline. Next, help your team create new testable builds as quickly and often as they want. Then, sprinkle in layers of automation and collaborative work. Continuous testing will look a little different at every company.
