Getty Images

Retail companies gain DORA metrics ROI from specialist tools

DORA metrics and other measures of engineering efficiency are popping up in add-ons to existing DevOps tools. But third-party vendors added more value for Puma and Sensormatic.

Two companies in the retail industry refined their DevOps processes with tools that offered suggestions for improvement and automation based on DORA metrics and other engineering efficiency measures.

Google's DevOps Research and Assessment team identified four key DevOps metrics in its annual Accelerate State of DevOps Report beginning in 2018: deployment frequency, lead time for changes, time to restore service and change failure rate. A fifth category, reliability, was added in 2021. Since the DORA survey was established, reports on its metrics have become commonplace within DevOps tools, from specialized vendor products to broader software suites such as Atlassian Jira.

In the last two years, DORA and other engineering efficiency metrics such as the SPACE framework have increasingly become a feature of those broader tools rather than the basis for a separate marketplace of vendors. But customers of specialists Sleuth.io and Propelo -- prior to the latter's acquisition by Harness -- found advantages in separate, focused products, such as in-depth data gathering, accuracy in data processing techniques, and recommendations on how to get better DORA metrics results.

Michael Gaskin, senior DevOps manager, global ecommerce, PumaMichael Gaskin

For athletic wear company Puma, DORA metrics reports in Jira weren't as mature in 2021, when the company's global ecommerce team chose Sleuth. Atlassian's Compass product, added in 2022, may also up the competitive ante for Sleuth. For now, however, Puma is sticking with the startup, which was founded in 2019 by former Atlassian employees.

"They have the hounds of hell upon them right now because everyone else is figuring out how to calculate DORA metrics and how to do deployment-related stuff," said Michael Gaskin, senior DevOps manager for global ecommerce at Puma. "But since they have a good team and they are agile and able to release things quickly, that bodes well."

Puma taps Sleuth to boost DevOps results

Gaskin took over Puma's ecommerce DevOps in 2021, as the company adopted a new headless architecture for its online storefront application. The headless architecture shifted the company away from a monolithic application to separated front-end and back-end components connected via API. This meant more frequent software deployments to different environments, such as the company's website and mobile applications. Gaskin said he wanted a data-driven way to show the company the benefits of the architecture change.

Sleuth gathered precise DORA metrics data from multiple workflow stages throughout the software delivery lifecycle. That meant Gaskin could pinpoint the specific causes of bottlenecks regardless of whether they were documented elsewhere.

"[Sleuth] could tell you the real truth of what was released and not an approximation of what was released based on what someone wrote down in Jira or Confluence," Gaskin said. "You could sometimes find things that you wouldn't otherwise know about."

[Sleuth] could tell you the real truth of what was released and not an approximation of what was released based on what someone wrote down in Jira or Confluence. You could sometimes find things that you wouldn't otherwise know about.
Michael GaskinSenior DevOps manager, global ecommerce, Puma

Two features added since Puma's initial purchase of Sleuth have Gaskin staying the course with the third-party vendor rather than moving to Atlassian's built-in features. One feature, Sleuth Actions -- which was launched in beta in January 2021 and is due for general release in the next few months -- kicks off DevOps pipeline automation based on DORA metrics results. The other is a recently released work-in-progress feature.

Puma was among the first customers to deploy both Sleuth Actions. It uses the feature to update specific Jira fields based on releases to each environment, including one-time URLs generated for each of its preview environments so that business users can test new features before they're released.

"We have to make it very obvious for them where they should go, [but] the preview environment URLs are somewhat dynamic. They have a number in them that changes each time," Gaskin said. "Sleuth was able to put that review environment URL into the Jira issue [automatically]."

Sleuth's work-in-progress feature expanded Gaskin's visibility into ongoing hidden issues within DevOps pipelines.

"As soon as I opened up those work-in-progress metrics for the first time I saw that there was this giant pull request out there that somebody had been working on for, like, three months, with 300 commits on it," he said. "I never would have really known about that if I hadn't spotted it in those work-in-progress metrics."

Gaskin said he uncovered this pull request too late to remove it, but in the future, he hopes work-in-progress metrics will help him intervene earlier in similar situations.

Sensormatic revved up velocity with Propelo

During the COVID-19 pandemic health emergency, Sensormatic, which makes sensors and other internet of things devices for retailers, had to adjust quickly to the flight of brick-and-mortar retail customers to online services. Now it must switch gears again to keep pace with the tech industry's AI craze.

About 18 months ago, Sensormatic bought software from Propelo, prior to Propelo's acquisition by Harness. It had managed to get a small social distancing and mask detection app out to its customers within eight weeks when the pandemic hit. But the company wanted to move even faster on developing future, bigger projects.

Subramanian Kunchithapatham, CTO, SensormaticSubramanian Kunchithapatham

The company brought in Propelo as it adopted a new internal process called Voice of the Customer that quantifies feedback from multiple end users and translates it into a priority list of new features and products, said Subramanian Kunchithapatham, CTO at Sensormatic.

"[When] we get some Voice of the Customer feedback for an application, customers expect that application to be ready for pilot in three months' time," he said. "When I'm working with a time constraint like that, I need to have end-to-end visibility into the team that is working on that application and how they're doing."

Like Sleuth, Propelo, now part of Harness Software Engineering Insights, can take action in software development pipelines based on DORA, SPACE and other DevOps metrics. But Sensormatic primarily uses Propelo's reports to give engineering managers feedback on where teams are blocked, so that they can intervene, according to Kunchithapatham.

"If the engineering manager who is working on the application has the visibility that, 'Hey, for this particular application, this team is stuck on code reviews,' that is where the can he or she can intervene and then bring in the right expertise to do the review so that that application can move as planned," he said.

Engineering managers can also get an overview of all the product development teams they manage, and reprioritize projects or reset project deadlines according to the progress they're making. Going forward, Sensormatic is interested in exploring a Propelo automation feature that enforces data hygiene and process adherence automatically within DevOps environments.

Sensormatic didn't use the Harness CI/CD platform prior to the acquisition. Instead it uses Spinnaker, and favored Propelo in part because of its compatibility with the DevOps pipeline tools it already had in place. This is still supported under Harness, but Kunchithapatham said Sensormatic is open to considering the Harness platform.

"We're looking forward to seeing their integration plans," he said. "I'm sure that they're going to open up a lot more efficiency and opportunities for customers like us."

Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.

Dig Deeper on Agile, DevOps and software development methodologies