People brand-new to DevOps often come in with a negative bias, particularly when it comes to software testing. Automation is a large part of DevOps for testers, but it does not eliminate the need for humans. What a move to DevOps does mean, though, is that testers must prepare to rethink this classic role. Testers must become risk managers. The testing work will not go away, but who does which job and when will shift.
How DevOps will change roles
As early as 1999, the advent of extreme programming suggested the end of the testing role in favor of continuous pair programming. At that time, a tester was more of a checklist-follower, and the thought was that automated tests written by a pair could replace that role. Since then, other ideas have evolved. Those include a single tester per developer pair, one test "coach" for a much wider group or a test team that assumes some of the build, merge and CI activities. Another option is to have very few testers but allow a programmer to play the "test" role if there is no tester available. Finally, some software organizations that support a wide variety of platforms (think Android, iOS and web) have turned to crowdsourcing and advanced beta programs to find bugs. In those cases, a few testers organize and oversee the test activities.
In her book, A Practical Guide to Testing in DevOps, Katrina Clokie wrote: "An organization might consider removing a tester if they believe that testing activities can be conducted by other roles within the development team or by the users themselves in the production environment. Though there are no definite rules for determining whether a tester can be removed, I see five areas to consider: context beyond the team, support for quality, team dynamic, measurement and bias."
Where the programmers take an idea from concept to cash, testers can follow along, if they can add value at every step along the way.
The common thread in all of these different ideas about software testing is a move toward one-piece flow. Once a story begins, it continues until the work is done, with only breaks for lunch, team meetings and going home at night. To do that, the technical working group needs all skills on deck -- from test to code to the database administrator to a little bit of operations.
When it comes to DevOps for testers, it is all about the value add. Testers can add build responsibilities or be involved in requirements, monitoring and rollback. Where the programmers take an idea from concept to cash, testers can follow along, if they can add value at every step along the way. If testers can contribute to one-piece flow, their role might be needed. As Clokie implied, if not, they might not.
Before you begin
To approach the DevOps for testers process for the first time, it's vital to analyze the current risk, how to manage risk, how threats might change with more frequent delivery and what the ideal end state will look like.
Of course, there is no ideal end state. Not only will the ideal end state change before the team reaches it, but once there, another higher mountain will loom. But an ideal end state gives the team a direction for improvement. If the team wants to ship more often, a regression test or a preflight check might be eliminated or reduced. If you eliminate checks, in turn, it might suggest a change in architecture to enable microreleases rather than working with a massive code base.
Another consideration with DevOps for testers is to determine who will be testing. If the entire team is involved in test, what is the mechanism to hold that together? If programmers will write all of the automated tooling, what skills will they need? If testers will write tooling, how can they integrate with the rest of the technical staff to enable one-piece flow?
One way to tackle this problem is to make a list of techniques and assign responsibilities appropriately. A better option is to take a further step back: Make a list of possible bugs, including security and performance, and decide how those risks will be managed. Then, address who will perform each activity in the end state.
Finally, consider the tooling and infrastructure that need to be built. Many teams start with the automation, but it is smarter to build the tooling (things like push-button environments) first. The DevOps for testers role at this point is to act as the customer or product owner for the build pipeline and monitoring systems.
Where to begin
At this point in DevOps for testers, the changes may feel overwhelming. Testers can't just switch to DevOps, especially if the build/test/deploy tooling infrastructure is not in place. Instead, identify a single target condition, or stretch goal, and iterate toward that.
The target condition is generally a combination of the largest gap. It is the gap with the most opportunity to add value to the process, but it is also the easiest to do. In other words, start with something that will give the most bang for the buck. Teams rolling out DevOps for testers generally start with infrastructure and tooling, which is fine as long as waiting for builds, deploys or environments is the single largest issue stopping the delivery organization.
Automation and tooling
Many organizations target the GUI for automated tests early in the DevOps process. The team needs to create a large number of tests, like a comprehensive web, to catch every bug. But if the problem is too many bugs, that's not a test issue; it's an engineering problem. A better plan is to focus on tools to accelerate the build and deploy of environments. The build tools make it possible to go from commit to test in five or 10 minutes.
The opportunities for DevOps testers
Exactly what DevOps for testers means and how it will work will be up to the organization. The first quick win is often the ability to create an environment that contains only the changes for a very small piece of work that can be tested. That step represents the desired one-piece flow from dev to test. Testers may also help with requirements at the beginning and act as a sort of junior product owner during the story. These new roles may be sufficient to keep their jobs relevant. Others move into the ancillary roles mentioned above. In any event, a DevOps for testers effort is a chance to get all technical staff to stop thinking of test as a boring checklist activity best done by someone else and instead to think of it as a continuous set of activities designed to reduce risk. It must touch every role.
Dig Deeper on Agile, DevOps and software development methodologies
