HashiCorp fends off challengers that sell ease of use with fresh infrastructure-as-code abstraction, while Terraform users grapple with a potential move to cloud-first services.
BOSTON -- HashiCorp has answers for IT pros seeking easier to use infrastructure-as-code and platform engineering tools, but for current Terraform customers, remaking existing environments comes with challenges.
Terraform Stacks, launched a year ago and moved to public beta this week, allows for grouping larger sets of Terraform configurations and their dependencies together than was previously possible and coordinating them across multiple infrastructure environments. For example, a Kubernetes cluster, an AWS virtual private cloud network and a Kubernetes application could be deployed and updated as a single unit.
With the beta release, Terraform Stacks also supports decoupling infrastructure components in new ways with deferred changes. Previously, Terraform would generate an error if users tried to set up a Kubernetes app configuration without first creating a Kubernetes cluster. Now the two can be created independently. Finally, Terraform Stacks supports orchestration rules, a feature that targets platform engineers with support for automatically approving new infrastructure deployments in development environments, for example, but blocking them in production.
"What if Terraform is … aware not just of one config and one environment, one set of state files, but of a richer topology, where you might have multiple layers of components?" said Armon Dadgar, co-founder and CTO at HashiCorp, during a keynote presentation at HashiConf Tuesday. "It really does feel like Terraform 2.0 in many ways."
With this update, Terraform added module lifecycle management features, also in public beta, that include change requests platform engineers can issue to application teams, prompting them to move to newer versions of infrastructure or application templates. Those change requests come with a new team notification system that can automatically send emails and warning messages through the HashiCorp Cloud Platform (HCP) Terraform UI to make teams aware of the change request.
HashiCorp CTO and co-founder Armon Dadgar presents a keynote at HashiConf 2024.
Another public beta feature broadens ephemeral workspace support to the project level, so that multiple workspaces can be automatically destroyed based on policy. HashiCorp's cloud-first push continues with a new custom Terraform provider, Terraform Migrate, that automatically generates HCP Terraform infrastructure code based on existing Terraform Community Edition HCL configurations.
HCP Waypoint is closely tied to these new features of Terraform. The developer platform service, made generally available this week, uses Terraform under the covers. New support for deferred changes, for example, means that Kubernetes applications and Kubernetes clusters can be separately configured while remaining linked together. But the Waypoint UI and a new API present catalogs of application templates, infrastructure projects and add-ons that shield application teams from code-level details.
Both products appear to be a response to newer competitors such as Harness, Pulumi and System Initiative that might tempt HashiCorp customers with ease of use, said Andi Mann, global CTO and founder of Sageable, a tech advisory and consulting firm in Boulder, Colo.
"Simplification, ease of deployment and collaboration with non-subject matter experts [is where] System Initiative but also some of the older startups are focused -- like Pulumi, for example, or Harness. They're really feature-competitive in a lot of ways," he said.
Terraform Workspaces vs. Stacks questions remain
The new features appeal to existing Terraform customers, some of whom have cobbled together homegrown versions of Waypoint for internal development teams. But the architectural shift from Workspaces to Stacks and the requirement to move to HCP for Waypoint is giving some of them pause.
While Workspaces will remain a part of Terraform long-term, in many cases, Stacks represents the next generation of Workspaces, according to Meghan Liese, vice president of product marketing at HashiCorp, in an interview with TechTarget Editorial. Workspaces allows multiple Terraform sets of persistent data, called Terraform state, to be associated with a single configuration, acting as a cookie cutter that can stamp out identical environments without having to configure a new Terraform back end. Stacks doesn't require environments to be identical and replaces homegrown scripts to manage dependencies between them.
"You heard Armon say on stage … that this is basically Terraform 2.0, and that comes out of this position of people … using this at great scale in very complex environments, and Stacks solves that," Liese said. "It's not surprising that people who are using this in a large organizational setting see Stacks as the future of what they want to do."
AJ Oller, assistant vice president of engineering at The Hartford, appears in a panel presentation with Kevin Tuffner (center), vice president at River Point Technology, and HashiCorp field CTO Sarah Polan at HashiConf 2024.
One such user, AJ Oller, assistant vice president of engineering at The Hartford, an insurance company in Hartford, Conn., has already migrated a large Terraform Enterprise environment to HCP Terraform. But the prospect of converting some 4,000 Workspaces to Terraform Stacks is a daunting one.
"To try and migrate that into a Stacks model is going to be pretty cumbersome. But that's where a nice utility, like a migration tool, would be helpful," Oller said in an interview with TechTarget Editorial.
Liese said HashiCorp heard that request from multiple customers this week but didn't say whether such a tool is planned.
HCP Waypoint stymies on-prem org
With these releases, HashiCorp continues its increased emphasis on cloud and commercial versions of its software. While a company blog post updated this week stated "certain aspects of Stacks" will be supported in the free Terraform Community Edition, Liese said that support will be minimal.
"At this point in time, it's available as something that you could grab and use and interact with. But you're still going to have that hard dependency on an HCP Terraform environment," she said.
On-premises Terraform Enterprise support is likely once Terraform Stacks reaches general availability. But for Community Edition, "the end goal there is that [users] can have some validation and a couple other tools that they can run locally. But then when you actually get to the deployment stage, that's an HCP Terraform thing," according to Liese.
Travis Rutledge, senior cloud engineer at Duke Energy, presents on his company's 'budget Waypoint' at HashiConf 2024.
Longtime Terraform Enterprise customer Duke Energy Corp. will wait for Terraform Stacks support on premises. But members of its platform team tested the feature in the cloud during a private beta period, according to Travis Rutledge, senior cloud engineer at the Charlotte, N.C., company.
"We have some complex infrastructure [managed with] run triggers and pipelines -- a bunch of random tools that chain together the Workspaces. But it's very hacky; it's all custom," Rutledge said in an interview. "Stacks gives us a standard way to do it in code that they all can follow."
However, Rutledge said his hands-on experience with Terraform Stacks suggested a potentially steep learning curve for direct use by developer teams. An equivalent to Terraform's no-code modules for Stacks would be welcome, he said.
That equivalent, according to Liese, is essentially HCP Waypoint. HashiCorp will gauge market demand for an on-premises version of that product. But for now, there isn't one available.
Rutledge said his team has cobbled together its own "budget Waypoint" using Python scripts and no-code modules. Those no-code modules are created with HCP Terraform, but the code can still be stored in Duke Energy's on-premises private Terraform Registry, Rutledge said.
"We don't have the opportunity or the ability to use Waypoint that's only available for the cloud offering," Rutledge said.
But later, during a HashiConf presentation, Rutledge told the audience, "Ever since Waypoint was announced, we honestly fell in love with the process of using no-code modules and kind of shifting that left, with developers not needing to always worry about Terraform."
An audience member suggested Rutledge try open source developer portal framework Backstage.
"That sounds great if you're starting from, like, square one," Rutledge answered. "What happens when we're already in production [with Terraform modules]? What happens with those modules [on] Day 3? With Terraform, you have the testing framework and the module deprecations."
We have some complex infrastructure [managed with] run triggers and pipelines -- a bunch of random tools that chain together the workspaces. But it's very hacky; it's all custom. Stacks gives us a standard way to do it in code that they all can follow.
Travis RutledgeSenior cloud engineer, Duke Energy
Another customer comfortable with cloud, GM Financial, has begun to roll out HCP Waypoint for use with existing HCP Terraform no-code modules, according to a presentation by Jeremy Myers, cloud engineer at the financial services company headquartered in Ft. Worth, Texas. Myers' presentation demonstrated the company's primary use of Waypoint so far: adding centralized secrets management with the HCP Vault Secrets service to existing applications.
"It's a system that allows us to host those no-code Terraform modules, really, and … we're able to extract all the [Vault Secrets] credentials away from the application team this way," Myers said in an interview following the presentation. "You could use many other types of [internal developer platforms] as well. But Waypoint provides a single platform for us since we're already leveraging HCP."
However, Myers said he looks forward to tighter integration between HCP Terraform and Waypoint, specifically so that if application teams' deployments fail, they can view the underlying Terraform plan to check for errors without requiring separate access to the HCP Terraform environment.
Liese said that integration is planned but declined to comment on timing.
Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.
