Pulumi's ESC, now GA, filled an automation gap in multi-cloud identity and permissions management for platform engineers well-versed in general-purpose programming languages.
After nearly a year in beta, a secrets management tool aimed at platform engineers shipped this week, and early adopters extolled its usefulness at an affordable price.
Pulumi's Environments, Secrets and Configurations (ESC), first unveiled in October 2023, adds a centralized interface to manage multiple secrets management tools alongside configuration data such as network settings and deployment environment variables. Secrets are bits of sensitive data such as passwords and authentication tokens. Some cloud-native systems issue temporary credentials that last only as long as one application's user session, and secrets management tools such as ESC can automate their rotation.
With the generally available version of Pulumi ESC, released this week, users can group related environments' security and configuration data into hierarchical projects, including versioning and tagging. A new syncing feature means ESC can respond to external trigger events, or trigger updates in external systems.
Finally, ESC now includes a Visual Studio Code extension and software development kits for commonly used languages and platforms, including Go, TypeScript, Python and a Kubernetes operator on first release.
ESC has competitors, including HashiCorp Vault Secrets Sync, Entro, GitGuardian, Teleport and BluBracket, but the inclusion of configuration and environment data, along with identity management for human users as well as machine-to-machine secrets management, differentiate Pulumi's approach. Pulumi rival HashiCorp, for example, offers such features but in separate products -- Boundary and Waypoint. It also uses a domain-specific HashiCorp Language (HCL), rather than general-purpose languages.
These design choices make Pulumi ESC suited to IT pros with a software development background who want to centralize management of infrastructure automation at scale -- in other words, platform engineers, said Torsten Volk, an analyst at TechTarget's Enterprise Strategy Group.
ESC was basically made for platform engineering.
Torsten VolkAnalyst, Enterprise Strategy Group
"It solves an actual problem where customers need a pragmatic solution for: How do I retain central control over security while at the same time giving developers self-service access, without them having to slide their own credit card?" Volk said. "ESC was basically made for platform engineering."
ESC early adopters: Toil reduced at the right price
Platform engineering teams at two companies that participated in Pulumi's ESC beta tests tried to put together their own homegrown answers to that set of questions but welcomed a more polished product from a vendor.
For one Pulumi ESC beta tester, using a general-purpose language with Pulumi's infrastructure-as-code product was more appealing than HashiCorp's HCL, given his background as an engineer at Facebook.
However, that tester, Jk Jensen, team lead and staff production software engineer at blockchain wallet vendor Mysten Labs, ran into a problem managing secrets and configuration at scale with Pulumi in early 2023. He brought up the issue with his company's account manager, who admitted him into very early tests of ESC.
"The problems we were having were [with] trying to evangelize Pulumi across the company," Jensen said. "[The platform team] had built some abstractions and done a lot of work to try and make it more accessible, but authentication and authorization was still an issue, where people would need all these random permissions across Google Cloud Platform in particular, and granting those permissions [individually] was challenging, especially in the case of [developers] wanting to move fast while the company's in prototyping and experimentation mode."
Liam White
Another ESC tester began using Pulumi's infrastructure-as-code tool more than six years ago for similar reasons. Pulumi infrastructure-as-code supported TypeScript, a language familiar to Liam White, senior software engineer and technical lead on the platform team at service mesh vendor Tetrate. Pulumi's TypeScript support also lent itself more easily to automating environment provisioning for multiple cloud environments behind a REST API than HashiCorp's HCL, he said.
Strong typing makes TypeScript infrastructure as code relatively easy to test using open source tools such as Zod, White said. Finally, it's well suited to use with an entirely Kubernetes-based infrastructure, where multiple versions of software-defined environments must be spun up quickly to test service updates and roll them out to customers.
"That was our original use case for doing our own hacky version of [ESC]," according to White. "I need to generate all of [these environments] and then feed them into Pulumi. And I can't just do it all in memory, because if it dies, I need to be able to rerun it from where it stopped, so I need some kind of persistence."
With the GA release, Pulumi set pricing for ESC, which was free during the beta period. That pricing starts at $0.50 per encrypted secret per month for the Team subscription, and $0.75 per encrypted secret per month for the Enterprise version, with volume pricing available.
For Tetrate, that amounts to a grand total of $1,200 per year, since it uses relatively few encrypted secrets despite the complexity of its infrastructure deployments, White said.
"Ultimately, anything that we don't have to run ourselves that costs less a year than a week's worth of U.S. engineering salary is pretty much a no brainer for us," he said.
However, White acknowledged that the equation is different when it comes to other Pulumi features that are available as only either a hosted service or via the Business Critical tier of Pulumi's subscription, which a company spokesperson said is typically priced "in the six figures."
White said he wishes he could get a more affordable self-hosted version of the Pulumi Deployments GitOps service, for example, but he isn't comfortable giving that level of infrastructure control to a hosted system. Pulumi ESC's appeal can also hinge on how strictly regulated a company is, White said.
"[HashiCorp] Vault has its own quirks, but there are reasons for that, which is often that you want to do something crazy, like FedRAMP, and Vault is probably more suited to that," he said. "Whereas if you 'just' need something like SOC 2, it's much easier to use something managed, like ESC."
Pulumi expands focus with ESC GA, Insights 2.0
With the general availability of ESC and a beta release of version 2.0 Pulumi Insights this week, Pulumi took a step beyond its original focus on infrastructure as code. With Insights 2.0, each of Pulumi's three major products will support being deployed independently of one another; Insights had previously been based on environments created with Pulumi's infrastructure as code, but the new version supports more environments, according to founder and CEO Joe Duffy.
"We're going to give you an asset inventory of everything you've got in all of your cloud accounts, [including] information about violations of security and compliance policies [through CrossGuard]," which is available as a feature for all products," Duffy said. "In fact, some of the [features] we launched [over the] last year, like automatic remediations, [will now] apply to infrastructure not managed by Pulumi."
The new visibility into more parts of the DevSecOps toolchain has Pulumi considering further expansion into software attestation, Duffy said.
With Insights 2.0, Pulumi is responding to some of the same general economic trends as IBM with its acquisition of Kubecost this week, said Justin Warren, founder and principal analyst at PivotNine in Melbourne, Australia.
"What I'm hearing from customers is that they are delaying [new] projects or they're rethinking their spend, particularly on cloud, now that interest rates are not zero," Warren said. "Rationalizing and being aware of what you have and if you're using it effectively is of great interest to a lot of customers."
Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.
Liam White
