HashiCorp Vault scalability updates target big enterprises
HashiCorp Vault 1.18 updates make it more suited to large companies, which the vendor is courting with a lighter cloud migration push than with Terraform.
BOSTON -- HashiCorp Vault updates this week reflected the vendor's large enterprise ambitions amid its pending acquisition by IBM.
HashiCorp Vault version 1.18, which reached general availability this month, featured updates to its underlying Raft database that make the security automation software work faster and more reliably. One change was an update to how Raft behaves in failure mode when a new leader server must be elected. In the past, network partitions in the Vault environment could cause a blip in connectivity during this process.
"Not an issue if you're [using] a low-scale cluster, but it can be an issue if you're doing tens of thousands of requests a second, and all of a sudden you're offline for a few seconds," said Armon Dadgar, HashiCorp co-founder and CTO, during a keynote presentation at HashiConf 24 this week.
Similarly, another Raft update, adaptive overload protection, makes HashiCorp Vault more flexible in how much concurrency it allows for requests based on available resources in the environment, and queues requests it can't accommodate. This means Vault clusters can perform faster and tolerate being overloaded with requests at high scale.
"We're on [HashiCorp] Consul for the secrets engine, the back end for Vault," said Dale Ragan, principal software design engineer at SAP Concur, in an interview with TechTarget Editorial this week. "We're going to be moving over to Raft. … [Adaptive overload protection] is some of the maturity I was looking for as we're starting to make that adjustment."
SAP Concur's switch from Consul to Raft will reduce the number of separate moving parts engineers must manage. But adaptive overload protection will be necessary in an environment where, each hour, 2,100 public key infrastructure (PKI) certificates are signed, 8,000 secrets retrieved, and between 12 and 14 database credentials automatically rotated, according to Ragan.
"Our biggest issue when we first started putting Vault out and using it [for] a wider audience … was just being DDoSed by our engineers," Ragan said.
Dale Ragan (right), principal software design engineer at SAP Concur, speaks with HashiCorp CTO Armon Dadgar during a keynote presentation at HashiConf 24.
HCP Vault Radar moves to public beta
The HashiCorp Cloud Platform version of Vault also received some attention from the company this week with the public beta release of HCP Vault Radar, based on HashiCorp's acquisition of BluBracket last year.
This release of the secrets scanning utility includes new integrations with code repositories and CI/CD pipelines to detect and prevent secrets from being exposed via commits, pushes, pull requests and merge requests during the early stages of application development. It also verifies whether secrets found in scans are valid with Vault secret correlation and offers remediation guides to remove secrets when they're found in places they shouldn't be.
The HCP Vault Radar beta also includes support for self-managed agents that can run on premises or within a private cloud and send only metadata to HCP. Given this support for on-premises agents, big Vault Enterprise customers such as Adobe are considering dipping a toe in HCP with Vault Radar.
"[It's] a pervasive problem, I think, for all organizations. … We have this Vault secrets management system. But are we covering everything? And where are our blind spots?" said Tyler Jacobsen, director of cloud operations and engineering at Adobe, during a HashiConf 24 presentation.
"The way that they've architected [Vault Radar] having the agent and not sending any [sensitive] data, I like that," Jacobsen added in an interview following the session. "That eliminates that barrier to entry a bit."
There are many alternative products available from other vendors that scan for exposed secrets in code and containers, including secret scanning built into GitHub and CI/CD pipeline tools. But the way Vault Radar hooks into multiple early stages of the development process was of interest to another Vault Enterprise customer, LPL Financial.
LPL also uses Prisma Cloud's infrastructure as code security scanning tool, which can test during the CI/CD process for security misconfigurations developers might make when using Terraform modules, but Vault Radar scans for exposed secrets when those Terraform modules are first created.
"You reuse Terraform modules, just like any other library," said Ashish Gupta, vice president of information security operations at the financial services company in San Diego, during a HashiConf 24 presentation. "If you take care of the security configuration in that library, nobody else has to think about it and fix it. … [By contrast,] we have something like Log4j, for example -- one security misconfiguration there, and the whole world is impacted."
Tyler Jacobsen (left), director of cloud operations and engineering at Adobe, speaks with HashiCorp field CTO Jake Lundberg during a breakout presentation at HashiConf 24.
HashiCorp cloud push gentler with Vault -- for now
Most of the HashiConf 24 speakers said they'd had the most success rolling out Vault Enterprise and features such as dynamic secrets by emphasizing slow evolutionary growth, rather than trying to cut over to the new system all at once. With support for on-premises Vault Radar agents -- in contrast to its cloud-only release of HCP Waypoint this week -- HashiCorp appears to be following a similar path with Vault Enterprise customers.
"When they're talking about things like PKI certificates, that's sort of the crown jewels for company," said Justin Lam, an analyst at 451 Research, a division of S&P Global. "That's one of the last things that people would want to have in the cloud."
But Lam predicted that will change, especially post-IBM acquisition, as company executives and investors push HashiCorp to move to a more lucrative annual recurring revenue model by selling more cloud services, Lam said.
You reuse Terraform modules, just like any other library. If you take care of the security configuration in that library, nobody else has to think about it and fix it.
Ashish GuptaVice president of information security operations, LPL Financial
Does this mean Vault Enterprise users will soon face tough choices about cloud migration? Lam said he doubts the decisions will be that tough in the long run, given how invested customers are in Vault.
"Vault is one of the stickiest things out there. It is literally the security marrow in the bones of an organization," he said. "I just wonder, what are the alternatives? If I balk at cloud, I also balk at all the other things that GCP, AWS and Azure provide."
In the meantime, HashiCorp continues to beef up cloud compliance, with planned roadmap support for FedRAMP and other regulations for large enterprises. HCP Vault Secrets, a scaled-down SaaS version of Vault, also added more advanced features this week that were previously reserved for Vault Enterprise, such as auto-rotation of secrets, finer-grained role-based access control support and support for streaming audit logs to tools such as Datadog and Splunk.
Jacobsen said his company will be keeping its "crown jewels" in house. But Ragan said the pending IBM acquisition could help HashiCorp ramp up HCP security, and he's open to considering HCP Vault for developer test environments to free his teams to focus on production.
However, HCP Vault Dedicated pricing per cluster per hour would be more expensive for SAP Concur than its existing Vault Enterprise environment, which is priced per active user.
"We have a prescriptive model of how we set up Vault. When we're making changes to it, it's not each engineer doing it. It's automated, so therefore, it's less people logging in," he said. "But if you have hundreds of engineers, you have hundreds of clusters."
Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.
Dig Deeper on Systems automation and orchestration
