Agile testing methods for multiple teams

A component strategy

Teams that can communicate in terms of interfaces, then define what interface they are using, can release at essentially any time. It sounds like a fairy tale, but that is how the Amazon.com homepage works -- it is a collection of smaller services combined with a larger container. Each of those smaller services, like "Related to items you viewed" and "Inspired by your shopping trends," can be tested in isolation. The container calls the latest version of the service, and the services can deploy to production as often as they want as long as their signature doesn't change. Even if the interface changes, the service can offer an older version-numbered service when requested, which is aged eventually. All this requires infrastructure work. Before you get to this, you may want to try some of the methods below.

Hardening sprints

This term does not mean writing code that doesn't work and then fixing everything at the end of the project. On a large "agile" project, that strategy, well ... it doesn't work. Instead, hardening is the period of testing interdependencies and large systems that require complex databases, performance and security tests that can only be done when the system is together, and so on. In a large Agile project, you might do this only every fourth or fifth sprint, toward the end of a program increment (see below). Of course, hardening is not ideal; with a component strategy, it might not be necessary.

The release train

Teams who used large, waterfall projects may want to begin by breaking things down to a slightly smaller increment. In the Scaled Agile Framework, or SAFe, this is called a program increment, or PI. A program increment typically consists of a release planning event, four development iterations, and one innovation and planning sprint. The SAFe guidance suggests the release train metaphor for these reasons:

• The train departs from the station and arrives at the next destination on a reliable schedule, which provides for a fixed cadence.
• All "cargo," including code, documentation, etc., goes on the train.
• Most people needed on the train are dedicated to the train, no matter what their functional reporting structure looks like.

The train gives the team the sense of timing, of what needs to be done and when. This allows the testers to have the same goal -- get the major releases out. Meeting together as a group to do release planning helps the team synchronize while the innovation sprint provides a little breathing and communication room.

The release manager

This is a singular person who coordinates the mini-project of getting to production. With multiple teams, the release manager probably has no technical responsibilities; all they do is coordinate details, manage the mini-project and make the key decisions to keep things moving forward. A very complex project may even have a release engineering team to coordinate complex builds and rollouts; sometimes this team is considered an "operations" team. Experienced professionals will point out that the release manager role exists to handle the pain of complex integration, to make it go away for the teams so the teams do not need to deal with it. A better choice might be to give the pain back to the team and force their way through it. I recommend some of both: Consider the release manager as a temporary, transitional role that will quickly learn to push coordination back into the teams. This allows the release manager to work on building self-service environments (the next bullet point) and cleaning the deploy/release process. Once that is automated, the release manager may find other work to do, as release management is no longer a full-time job, but instead a part-time job that can be rotated among team leads per program increment.

Discipline around builds, environments and source code

Multiple beta-like deploys to the System Integration Environment (SIE) per day is a great recipe for a SIE that is down a lot, perhaps always, which is only fixed during "hardening." Yuck. A self-service environment works more like this: Go to a webpage, check a build, wait five minutes, have your own Web server. Teams that have this sort of self-service dev and test environments can check their work before it goes to the SIE, increasing overall performance and helping get rid of that hardening sprint.

SCARE

The "Sustainable, Cultural Agile Release for the Enterprise" is an alternative to scaling that starts by finding the "heartbeat" within the large, multi-team project and accelerating that. That "heartbeat" is the core group of people working on the critical path; the single component on which the project hinges. In a large project that is the most important project of the company, a good part of the company’s future may rest on those 20 or so people. At that level, traditional Agile testing methods might work just fine. Meanwhile, the team heartbeat is the single thing holding the project back. Every day that the heartbeat is late, the project is late. Every day the heartbeat is early, the project could be early. That means a day saved by ten people could save hundreds or thousands of person-days. Once the first heartbeat team is built, you can declare victory; keep them around for the next project and you have an effective high-functioning team. Or slowly take volunteers to create additional teams. This means the team will be slowly moving toward Agile, a much smaller problem than "whole team" conversion.

Figuring your way out

When I talk to teams and people adopting Agile methods, they often want a single silver bullet. Just do X, Y and Z, and you'll be fine. The problem is that what works for one team might not work for another. More than that, what might work for your team in a few years, once you have some capability, might not work today. If a technique sounds like it might solve the problem, try it. If not, pitch it. When the team has the skill that it no longer needs the technique, the technique can go away.

For example: When the team can deliver components, the hardening sprint goes away. When the team can deliver continuously, the release train goes away. And so on. The end state could be continuous delivery to production with no significant "release candidate testing" step.

You might never get there, and that's okay. Walking toward continuous delivery and removing obstacles will mean faster releases and likely fewer defects -- which means less time spent on fixes and more time spent on new features.

Now that is something to shout about.