Without DevOps metrics, forget about DevOps glory

When your organization decides to break down siloes and orient its development and operations work around DevOps principles, it's reasonable to wonder if you are making progress on the right things -- or if you're making any progress at all. That's where DevOps metrics come into play.

An IT team is generally good at calculations. It can quantify uptime. It knows how many helpdesk tickets it handled in a given period. It's possible to compare this year's spending versus last year's to see if that decision to switch vendors saved as much as you were promised. And a legacy system that's finally put out of its misery is an accomplishment that's celebrated and marked on the calendar. Whatever it is you're doing in enterprise IT, it's almost always possible to download the results, measure the successes and graph the failures.

But what about metrics for DevOps and all that cultural change? How do you gauge accomplishments and tally wins and losses? This is where the shrugs come out. Maybe you'd like to know if this new way of working produces results, that's it not all just a big shakeup to collective comfort zones simply to make some senior execs feel better about their innovation strategies. But more than likely, you don't know. An optimist will trust her gut. A doubter might roll his eyes.

The trouble with DevOps is that it is a process that's ongoing. It provides no endpoint to contrast with the start point. Is this way the right way? What about that way? How can you even tell? Plenty of people will point to these seemingly unanswerable questions as the essence of the DevOps problem: It's just too difficult to know what works and what doesn't.

Or is it?

Before we throw up our hands and toss the whole thing into Gartner's trough of disillusionment, let's give DevOps metrics another look. With a clear-eyed approach that measures results, even skeptics can get behind DevOps.

Find a few good metrics and ruthlessly follow them wherever they lead. As Anders Wallgren, CTO at Electric Cloud, put it: The only DevOps metrics that matter are ones that are both objective and actionable. "A metric that doesn't have an outcome and doesn't have an action that helps you achieve that outcome is generally an uninteresting and maybe even a bad metric," Wallgren said at the 2018 DevOps Enterprise Summit in Las Vegas.

The temptation to measure everything is understandable, but that can be the road to ruin.

"Pick something that you don't like," Wallgren said. "Pick something that drives you nuts. Pick something that takes too long. Pick something that fails too often. Just pick something and then figure out a way to measure that and drive a better outcome for that thing. And then move on to the next thing."

If you continue to find ways to get better, be it with mean time to recovery, release frequency or any number of other DevOps metrics, you should be able to deliver better software and keep your customers happy.

Adopting DevOps metrics does not mean you should count the lines of code produced. While that may be an objective measurement, it's not in any way relevant to outcomes. Concentrate on a few things that help you make better decisions, experts insist, even if those items don't seem like they make an enormous difference to IT overall.

See how much better you can become at failure detection, or how much quicker you can deliver a VM. These may not be the big moments you envisioned when you and your colleagues got into DevOps. But forget about those seismic shifts in culture, where your organization experiences cross-functional bliss and every day ends with a group hug. Figure out what you want to do better, and identify the DevOps metrics that will tell you if you are on the way there.

Maybe you'll decide that full-scale DevOps isn't workable in your organization or that the gains aren't worth the pains. To make an informed decision, though, you're going to want to measure the right things in the right ways.