OpenTofu forges on with beta feature that drew HashiCorp ire
Defying a HashiCorp cease and desist, OpenTofu 1.7 beta ships with the removed blocks feature and client-side state encryption support long sought by the Terraform community.
OpenTofu's latest beta release defies the original creators of the infrastructure-as-code software it forked in two ways: It retains a feature HashiCorp accused it of illegitimately copying and advances a feature maintainers say is the first users can't also get from Terraform.
OpenTofu began after HashiCorp declared in August that it planned to ship future versions of its popular cloud-native infrastructure automation tools under a Business Source License (BSL), citing unfair competition from companies that based products on its open source code without contributing back. In the ensuing controversy, a consortium of those competitors and others in the open source community launched a fork of HashiCorp's Terraform infrastructure-as-code tool based on its most recent open source release. The project was subsequently admitted into The Linux Foundation, and its first production-ready stable release launched with version 1.6 in January.
Meanwhile, in November, HashiCorp published new Terraform code to its GitHub repository under the BSL that refined the process by which users can refactor Terraform configuration code without permanently destroying infrastructure resources saved in Terraform's state files. This was part of a redesigned process for "a new, faster, and less error-prone method" for revising Terraform deployments in version 1.8, according to a HashiCorp blog post. These features were based on new blocks of Terraform code called removed blocks.
In February, a GitHub pull request proposing "Add support for removed block" showed up in OpenTofu's repositories, and the feature was added to the alpha release of version 1.7 in March. On April 3, HashiCorp sent a cease-and-desist letter through its lawyers to OpenTofu, claiming that OpenTofu violated its BSL terms by copying the removed block code.
OpenTofu responded publicly on April 11 with a detailed refutation of those claims, including a line-by-line source code analysis, which stated that its removed blocks were developed based on the open source version of Terraform, not the BSL version. A third-party allegation that OpenTofu had copied the BSL code -- published in an industry publication on April 3 -- was also updated with an editor's note stating that it now appeared that had not been the case, and its author issued a public disavowal on social media.
Sebastian Stadil
The two removed block features achieve the same goals from a user perspective -- changing resource configurations without deleting resource state -- but took functionally different paths to get there, said Sebastian Stadil, founder and CEO at Scalr and an OpenTofu core member, in interviews with TechTarget Editorial this week.
"Generally, we prioritize features based on community feedback and needs," Stadil said. "Some of them are similar to what Terraform provides, and some are exclusive for OpenTofu users."
Stadil said there has been no further communication with HashiCorp since the public response by OpenTofu, and that OpenTofu is in the process of responding to takedown notices sent to GitHub about OpenTofu repositories by HashiCorp under the Digital Millennium Copyright Act. So far, no repositories have been removed, he said.
HashiCorp reps did not respond to a request for comment this week.
OpenTofu 1.7 throws down state encryption gauntlet
OpenTofu version 1.7, released in beta Thursday, also contains the first of a planned set of "flagship features" OpenTofu will support that Terraform doesn't, Stadil said.
"With OpenTofu 1.7, you get the better option for free and open source, versus with Terraform you get the lesser option for a fee," he said.
This feels like another case where HashiCorp is ignoring the community because of business reasons, rather than there being true technical concerns.
Robert HafnerPast Terraform contributor and author of 'Terraform in Depth'
That feature, client-side state encryption, has been requested by members of the Terraform community since 2016, but has not been incorporated into the HashiCorp project. HashiCorp's version supports the encryption of state -- a set of files that Terraform uses to map configurations to infrastructure resources and track metadata -- when hosted remotely on a cloud back end, including its own Terraform Cloud and those of cloud provider partners. Users can also mask specific data within state files using a "sensitive" flag.
"One experiment that has been attempted is allowing the user to provide a PGP key and a cipher text, and decrypting the value in the provider code before using it, storing only the cipher text in state," the documentation reads. "Even without comparing it to full state encryption, PGP key encryption has major drawbacks. Values encrypted with a PGP key can't be reliably interpolated, Terraform isn't built to provide a good user experience around a missing PGP key right now, and the approach needs serious modification to not violate protocol requirements for Terraform 0.12 and into the future."
This leaves locally stored state unencrypted, but the Terraform documentation notes that strategy "was tailored to a time when Terraform's state had to be stored in cleartext on any machine running terraform apply." That is no longer the case since the introduction of remote state back ends.
However, remote state encryption by back-end cloud providers means users can't separate access to state stores from access to the Terraform client or selectively encrypt certain files within those stores, which some highly security-conscious users would prefer as an added layer of protection.
"It's the difference between you encrypting something, then giving the encrypted something to me to store, versus you giving it to me unencrypted, and I encrypt it before storing it," Stadil said. "In the former case, I never see the unencrypted data; in the latter case, I do."
State encryption pros and cons
Discussions and debates about client-side state encryption in both communities included concerns about users potentially losing access to Terraform state if they lose access to client-side encryption keys. But OpenTofu's approach lets users specify a fallback configuration for decryption, and some users said that risk has been overblown while ignoring the potential benefits.
"The state encryption provided by OpenTofu encrypts the entire state file, not just individual values, and does not rely on providers to do so," said Robert Hafner, a past Terraform contributor and author of the book Terraform in Depth. "This feels like another case where HashiCorp is ignoring the community because of business reasons, rather than there being true technical concerns."
The feature also drew interest from another IT pro whose organization will stick with Terraform Community Edition while OpenTofu remains new, but who has closely followed its progress.
"We use GitLab's self-hosted version [of Terraform state], so we can configure the data store behind GitLab, [where] we manage security access and encryption," said Phil Fenstermacher, manager of systems design and architecture at William and Mary, a university in Williamsburg, Va. "[But] if someone manages to get access to the files on the back end, they'd be able to read the contents as plain text. We'd like to use client-side encryption to encrypt sensitive data within the file because we can then use different encryption keys for different projects, as opposed to one for everything behind GitLab."
Beth Pariseau, senior news writer for TechTarget Editorial, is an award-winning veteran of IT journalism covering DevOps. Have a tip? Email her or reach out @PariseauTT.
Dig Deeper on Systems automation and orchestration
Sebastian Stadil
