When IT ops teams centralized infrastructure automation behind developer-friendly self-service interfaces with DevOps platforms, developers weren't the only ones who benefited.
DevOps platform teams that took a product-oriented approach to providing developers with IT services also reaped cost and security benefits for IT operations.
DevOps platforms hosted as a managed service for internal developer customers have gained momentum as DevOps matures, especially amid the upheaval and budget restrictions of the COVID-19 pandemic. In many cases, this strategy also reflects a common staffing limitation in which so-called "full-stack developers" -- once the DevOps ideal -- have proven hard to come by.
Instead, recent industry research indicates that enterprise IT organizations that maintain some separation between developers and operators, but encourage efficient collaboration between them, tend to be the most effective as measured by DevOps metrics such as speed of software deployments and the number of deployments per day.
Many such high-performing organizations shift from a project-based mentality, where IT teams gear their work toward a specific short-term goal, to a product-based mentality, in which one group -- developers -- is cast as internal customers and the other -- IT ops and site reliability engineering -- as internal service providers focused on optimizing customer experience long-term.
In 2018, automotive equipment manufacturer Oshkosh Corp. began such a transition, recasting its IT department from a subset of its finance organization to a product division that serves internal customers.
Jared Petersen
"We embarked on this journey to transition our culture from a necessary evil and a cost center to operating more like a business," said Jared Petersen, senior director of digital technology strategy and portfolio management at the company located in Oshkosh, Wis. "We can try to create more of a free market-type environment and naturally provide our business customers and our end customers [with the] products and services they need."
This approach may seem to prioritize developer experience over IT ops benefits but it led to improvements for both at Oshkosh, Petersen said.
DevOps platform integrations spur IT ops efficiency
Two software products form the basis for Oshkosh's IT services platform. The first is ServiceNow's Now Platform, which performs a broad swath of functions from change management to application portfolio management and business services offered as digital products such as robotic process automation (RPA) and IT infrastructure provisioning. The second is Apptio's ApptioOne IT financial management software.
"Our digital product owners will have regular meetings with the business customer lead ... and say, 'Hey, here's what our unit price is, here's where we're seeing consumption increases [and] here [are] areas that we believe ... we could improve or better collaborate on,'" Petersen said. "We've customized a report that automatically comes out of Apptio to do that."
Apptio's cost analysis helps Petersen's team market its ServiceNow platform to internal customers and generates return on investment data to demonstrate its benefits. The platform's internal customers expanded to include developers who also worked with Azure DevOps last year, Petersen said.
We can now say with confidence and show back ... the value proposition of what we delivered and its costs. It brings confidence to the new investments that the business is making in our products.
Jared PetersenSenior director of digital technology strategy, Oshkosh Corp.
"We can now say with confidence and show back ... the value proposition of what we delivered and its costs," he said. "It brings confidence to the new investments that the business is making in our products."
The Apptio and ServiceNow combination can troubleshoot problems when they come up, such as budget overruns due to IT misconfigurations, which also reduces the operational cost required to run IT services.
"If I'm the leader of our [ServiceNow Intelligent Automation Engine] capability, and I'm looking at the [total cost of ownership] of my RPA service, I know specifically who to talk to if my network costs have increased," Petersen said.
Earlier this year, Apptio released a set of bidirectional data integration modules for the ServiceNow Application Portfolio Management app that correlate costs in Apptio with detailed ServiceNow IT infrastructure usage data.
This will help Petersen's department report more precise infrastructure-related costs for internal IT services customers, including developers who deploy to the Azure cloud through that platform, he said.
"A year into our journey we followed up with Apptio and said, 'The area we feel a bit vulnerable, or we'd like to automate better, is the integration of how many [service] units the business is consuming,'" he said.
For now, there's no similar direct integration between the ApptioOne IT financial management tool Petersen's team uses and Azure DevOps, although ApptioOne customers can create custom integrations. Petersen said he'd like to see such a tie-in built into the product, rather than creating a custom integration, to give developers insights on application deployment costs.
Still, shifting to a DevOps platform offered as a product with detailed chargeback has already helped Oshkosh make better use of the ServiceNow portfolio, Petersen said.
"In 2018, our project portfolio value for ServiceNow was less than $10 million in terms of the annual [operational impact]," he said. "Today, that's more than $30 million -- we've seen that portfolio value almost triple."
SANS reaps security upgrades from platform shift
SANS Institute, a cybersecurity training and certification company based in Bethesda, Md., was no stranger to IT automation and DevOps when the COVID-19 pandemic struck the U.S. in March 2020. But the pandemic forced staffing cuts that prompted SANS to rethink its DevOps platforms, which streamlined its IT security processes.
Previously, SANS Labs, a division of the Institute that manages online cybersecurity training courses, had used a mix of IT automation tools to provision temporary infrastructure for students. These included infrastructure-as-code (IaC) tools such as HashiCorp's Terraform and AWS CloudFormation; CI/CD tools such as CircleCI, Mergify and Code Climate; and assorted bash scripts and AWS Lambda functions.
In the meantime, SANS DevOps engineers in the company's security awareness business unit had begun to use an IaC tool from Pulumi that supports application programming languages familiar to developers, such as TypeScript and Node.js. When SANS leadership called for a reduction in IT staff and more workflow efficiency amid the pandemic, this team replaced its own mixed toolset with Pulumi and GitHub Actions for CI/CD. In early 2021, the security awareness unit's DevOps engineers were called in to advise the SANS Labs team on a similar refresh.
Tyler Mulligan
"As demand increased, the manual processes that were more acceptable when the requests were not as prevalent were fine," said Tyler Mulligan, one of the SANS DevOps engineers that implemented the new platform. "But as they started to scale and COVID hit, we had to serve more on-demand needs, and [the] system that they built out to integrate with their continuous integration tools was no longer really working for them."
Mulligan and his colleagues had used Pulumi to create a homegrown developer self-service interface in TypeScript that included IaC templates that developers could adapt for their applications, saving them the time and effort of learning a separate tool. An Automation API Pulumi introduced in April further streamlined the developer experience for SANS Labs because DevOps engineers could replace the homegrown self-service system with a built-in, developer-friendly REST API.
The DevOps team also pre-built Pulumi Policy Packs to enforce security and compliance standards under the newly centralized system, which deploys infrastructure and application code through GitHub Actions.
Consolidating the number of tools developers use and standardizing the developer experience led to more consistent security policy enforcement in the new system than had been possible with the previous toolset, Mulligan said.
We can't uphold the same level of [security] standards in a bash script that we can in testable TypeScript.
Tyler MulliganSenior DevOps engineer, SANS Institute
"That's one of the stickiest parts of operations for me personally, that we can't uphold the same level of [security] standards in a bash script that we can in testable TypeScript," he said. "[The Release Manager GitHub Action] allows us to enforce our standards in the pipelines while giving the developers less work because all they need to do is push the branch when it's ready. And then if all our checks and balances pass, it gets released."
Mulligan's team now hopes to expand the use of this system to more SANS business units and developers working in other programming languages, including Python.
"We're in the process of trying to see if that will foster additional adoption of Pulumi, by providing more familiar languages to developers," he said. "We can share a lot of the same tools that we write our applications with in Pulumi [to do] enforcement and security checks on [application] dependencies using one ecosystem, rather than being spread across multiple [tools]."
Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.
Jared Petersen
Tyler Mulligan
