Know the performance hang-ups of AWS' IaC tool

CloudFormation performance issues

CloudFormation is a native AWS IaC tool that makes service calls to AWS to provision resources for a stack, based on the template supplied by the user. Enterprises can deploy consistent IT infrastructure more rapidly with IaC than with manual procedures.

The biggest frustrations with AWS' IaC tool is troubleshooting errors and a lack of logic or conditionals, said Josh Quint, senior director of cloud solutions at ServerCentral Turing Group, a provider of managed cloud and data center services.

For example, error messages from a failed stack deployment are only helpful about half of the time, Quint said. The error messages are often too generic and don't point users towards the problem -- or worse, they point in the wrong direction, he added. 

Furthermore, CloudFormation can fail well into the stack creation process, which causes longer development cycles and more frustration. It may take several minutes, or even hours, to create a stack, only to have it fail at the last second, Quint said. Then, you usually have to delete the stack -- which can take a while -- and start over. In other cases, a stack doesn't fail, but simply times out for no clear reason.

AWS quells some concerns with CloudFormation roadmap

CloudFormation is not typically known for controversy, but online complaints began to surface in May 2019 regarding AWS' IaC framework. Users sounded off on a perceived lag in how CloudFormation is updated -- or not updated -- to work with other products on AWS. Some went further and assumed that this lack of updates implicates a general lack of organizational support for CloudFormation at AWS.

To remedy this disconnect, AWS released a public roadmap for CloudFormation. Also, AWS chief evangelist Jeff Barr posted a mea culpa to explain that AWS has focused the bulk of its effort on scalability rather than ensuring that all services on the platform are supported by CloudFormation. It's an imbalance the company is working to correct, Barr said.

However, this feature lag might not have been felt by enterprise AWS users. Jim Mercer, an analyst at IDC, said he hasn't heard of any complaints about CloudFormation from clients. Chris Gardner, analyst at Forrester, reported much the same.

AWS CloudFormation vs. Terraform

While it's unclear how much feature lag has impacted usage, it's certainly the case that many organizations have opted to go with third-party alternatives such as Terraform instead. There are several reasons for this, but the biggest one is that CloudFormation only works on AWS.   

"Most clients I speak to are using multiple public clouds and for that, Terraform is better because it's cloud-agnostic," Forrester's Gardner said.

When you compare the AWS IaC tool with Terraform, each one has its strengths. For example, it's easier to add and replace resources in a stack with Terraform, and its planning feature provides details on exactly what will change, said Mark Runyon, principal consultant at Improving - Atlanta.

Users should go with CloudFormation for smaller projects that need to get up and running quickly, since the JSON and YAML code formatting are generally easier to work with [than Terraform's domain-specific language], Runyon said.

Where AWS IaC goes from here

The public roadmap is a good step for the future of CloudFormation, but it may not address the larger problem with the service.* CloudFormation relies on what is really a templating language, not a scripting or application programming language, Quint said. And as stacks get more complex and require more flexible automation, users need to perform basic logic and variable manipulation within these IaC templates. This functionality, which requires more of an application programming language, would also cut down on the number of lines of code and make the stack less error-prone, he added.

So far, AWS has included some extra verbiage to error messages and a little extra validation before a template is deployed. They have also added the ability to cancel a stack deployment, so you can avoid some of the time-out issues, but there's still no good way to know how long a stack should take to deploy, Quint said.

And while there have been ways to get Mappings and ImportValue functions to work as variables and do some basic logic demonstrated by AWS, these tend to be complex and not particularly user-friendly.

"There's plenty of sample code out there for custom resources that allow CloudFormation to kick off external functions and wait for the return value, but this seems like it's akin to a Rube Goldberg machine," Quint added, implying that CloudFormation demands many steps to accomplish a simple task.

There has been a push to make it into a service that can not only template AWS cloud infrastructure but also configure instance OSes and even deploy applications, Quint said. But these tweaks can't turn CloudFormation into something it's not, and AWS can't really revamp CloudFormation and still preserve backward compatibility, he added. Long-term, there could be an opportunity for a higher-level AWS IaC language that's native to the platform that serves as a counterpart to CloudFormation.

For now, users should recognize the service's limitations and try to keep their CloudFormation stacks focused on infrastructure instantiation and AWS resource configuration, Quint said. Users can depend on other tools for OS configuration management, such as Red Hat Ansible or Salt, and CI/CD tools for the application deployment. 

Editor's note: This article was updated to clarify Josh Quint's comments on the future and viability of the CloudFormation public roadmap.