WavebreakMediaMicro - Fotolia

Pulumi cloud engineering boosts dev infrastructure control

A Pulumi early adopter says this week's cloud engineering update will save time and resource costs with more support for developers to test infrastructure-as-code plans locally.

The latest release of an infrastructure-as-code tool from Pulumi solidifies its cloud engineering integrations and expands developers' power over infrastructure resources.

Pulumi, founded in 2017, initially created a developer-friendly interface in familiar programming languages such as Python and Node.js for the open source version of HashiCorp's Terraform infrastructure-as-code tool. Infrastructure as code, in which software-defined resources are configured and provisioned through source code repositories and pipelines, has become popular alongside cloud-native architectures such as Kubernetes container orchestration and GitOps.

Terraform, by contrast, uses a domain-specific language; other infrastructure-as-code and configuration management tools such as Kubernetes Helm charts and AWS CloudFormation templates use YAML and JSON, which are less familiar to software developers.

Keith RedmondKeith Redmond

"I've had 50 cloud developers working with Pulumi [in the last year], and maybe two or three of them had cloud experience," said Keith Redmond, vice president of SaaS engineering at Fenergo, a financial services software maker and SaaS provider based in Dublin. "But I can count the number of misconfigurations [in production] on one hand because Pulumi breaks the build [when they occur] -- it's not letting us make those mistakes."

With Pulumi version 3.0, which shipped this week, the vendor swapped out the Terraform code it had used to control cloud resources for its own native integrations with public clouds. Its new integration, which the vendor calls a cloud provider, for Microsoft Azure is generally available now, and a Google Cloud Platform provider is available in beta. New AWS cloud providers will follow later this year.

Redmond said the updated cloud engineering integrations will eliminate the last few instances where his team had to use CloudFormation or scripting workarounds to control cloud resources Terraform providers didn't support. These include AWS Step Functions and X-Ray, which the company uses with AWS Lambda serverless computing, as well as the Amazon Neptune graph database.

"The Terraform [open source] providers were a slight source of frustration," Redmond said. "With the native providers, I'm happy Pulumi has more control over its own destiny."

Pulumi Automation API enhances local testing

This Pulumi release also includes a new Automation API that makes infrastructure resources available as a library, which developers can embed within their application code, and test alongside it.

Pulumi's native cloud providers underpin shareable Pulumi Packages, blocks of code that describe sets of infrastructure resources. The Automation API directs how they are within developers' applications. This API can be treated like any other library, which means developers can easily test infrastructure-as-code plans on workstations before they're deployed to staging environments.

The new API will save more time and effort for Fenergo's development teams, Redmond said.

The future of cloud engineering will see developers looking to work more locally…Automation API will let them test and tear down infrastructure before it's released to any real environment.
Keith RedmondVice President of SaaS engineering, Fenergo

"The future of cloud engineering will see developers looking to work more locally," he said. "It's expensive and time-consuming to work against the actual cloud. The Automation API will let devs test and tear down infrastructure before it's released to any real environment."

In future releases, Redmond said he hopes for more help from Pulumi in meeting Fenergo's regulatory compliance requirements, through more detailed workflow approvals and audit trail visualizations.

"We can get the data we need, but we'd like to see more detailed audit trails for individual resources and build approval chains into Pulumi," he said. Fenergo has an approval gate when its GitHub Actions CI/CD pipeline hands off to Pulumi and after the Pulumi stage before cloud infrastructure is spun up, but he said he'd like to add more such steps as Pulumi executes.

Pulumi integrates with third-party CI/CD providers for such workflows but has plans to build more fine-grained controls directly into the product on its midterm roadmap, according to CEO Joe Duffy. Pulumi 3.0 contains the product's first set of dashboards, and the vendor's roadmap also includes better filtering and searching across resources and audit logs, Duffy said.

Meanwhile, Pulumi has taken the place of DevOps engineers and IT ops specialists for Fenergo, which Redmond said has been crucial for the company as it transitions from an on-premises software vendor to a SaaS provider.

"I'd need a team of five or six DevOps engineers in place if I didn't have Pulumi," Redmond said. "That's more than a million Euros [in salary] that's gone to product developers instead."

Pulumi's competitive questions remain

Though Pulumi's infrastructure-as-code tools were previously based on Terraform providers, the company has taken an aggressive stance against the infrastructure-as-code leader that locked up the Kubernetes infrastructure-as-code market before Pulumi was founded.

Torsten Volk, Enterprise Management Associates Torsten Volk

"Kubernetes and Terraform go together most of the time," said Torsten Volk, an analyst at Enterprise Management Associates. "They're like an old married couple at this point."

Pulumi distinguishes itself with a strong focus on appealing to developers and working in their preferred programming language, while Terraform approaches infrastructure as code with ops and DevOps specialists in mind. Developers can use Pulumi Packages in their preferred programming language even if they were written in another, for example.

This differentiation will have the most appeal to companies such as Fenergo that want software engineers to own the full application stack, including cloud infrastructure, but it remains an open question how widespread such scenarios will be.

"The biggest challenge here is how to get developers to use and care at all about policy and provisioning [tools]," Volk said. "Pulumi pops up in a lot of different places in our market research, but they'll have to step things up from a competitive perspective to get more mind share against Terraform."

DevOps and full-stack engineering are popular buzzwords, but in reality, many developers don't want to deal with managing infrastructure, Volk said. Meanwhile, HashiCorp's recent Terraform-focused partnership with Cisco will raise its profile further among traditional enterprises.

"I don't hear of many people using [Pulumi] across their whole company yet," he said. "But a lot of people do complain about Terraform, when they have to code things over and over again."

Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.

Dig Deeper on Software development team structure and skills