Are you using the right DevOps metrics in your organization?

Mainline branch stability

When it comes to deployment, the key factor is mainline branch stability, because that tells a company how quickly and how often to continuously integrate, Rose said. According to the survey, 80% of organizations keep the mainline branch stable 90% of the time. Highly ranked Alexa companies, those in the 95^th percentile, kept their mainline branch stable 99.9% of the time. "Lots of people get over their skis and want speed, speed, speed," he said. "We want you to move fast, but if you're not in a good stable place to do that, you can inject a lot of chaos into the project."

Jim Rose

The report has some ideas to make the mainline branch more stable. Best practices include healthy test suites, the use of feature flags and a solid strategy for failure recovery. Another suggestion is to make sure the entire team understands the costs of failure. Rose noted that some companies actually send DevOps teams to the field to watch end users and develop a real-world understanding of the problem.

DevOps deployment frequency

DevOps metrics related to how fast a company deploys are also linked directly to business success. The survey said 80.2% of organizations deploy in under 15 minutes. But the top 5% of the Alexa ranked companies deploy in an average of 2.6 minutes. And the slowest deployment time, in the bottom 5% of the Alexa scoring, was 36.1 minutes.

The DevOps metrics-business success connection is also true when it comes to deployment frequency. The survey showed 75% of companies roll out top projects less than 13 times a week. But for those in the 95^th percentile of Alexa ranking, it's a completely different picture. They deploy 32 times a week -- a number that is over 24 times more often than the bottom 5%.

Stability first

But Rose stressed deployment frequency is not the total picture of a healthy DevOps organization. Companies without strong testing in place may end up deploying more often, simply to fix the bugs.

Consider these DevOps metrics: Companies that spend more than 12 minutes on a build deploy a median of 8.3 times a week but also have a much higher percentage of main branch instability (2.7%).

"Once you can get to a firm stable footing, you can branch out to microservices and polyglot development and other things," Rose said. "But you've got to get to stable first."