GitOps path puts CI/CD ahead of infrastructure automation
Which comes first -- automated app deployment pipelines, or automated infrastructures on which to deploy? At GitOps shop Procore, the answer is emphatically the former.
Is GitOps the end of the digital transformation journey, or the beginning? Construction management software maker Procore Technologies has a firm answer, and it's different from that of other enterprises.
Some forward-thinking DevOps shops see GitOps -- in which version-controlled code repositories automatically update production applications and infrastructure -- as a future goal, but that's where Procore started in mid-2016. Advanced infrastructure automation techniques such as container orchestration and service mesh remain on the company's to-do list, but it already operates a CI/CD pipeline that automatically deploys code to production roughly every 90 minutes.
"We wanted to rapidly change and show our customers new features, possibly behind feature flags but still getting into production, because that's the only time when it actually is valuable," said Kyle Espinola, senior software release engineer at Procore, located in Carpinteria, Calif. "This is the mechanism that allows us to give [customers] the features they need to do their jobs…at this point we default to releasing code to production as much as possible."
That focus on business value made Procore's priority the GitOps pipeline. Code committed to a master GitHub branch automatically kicks of continuous integration tests and Docker image builds in CircleCI, and a homegrown tool based on Capistrano deploys those Docker images to AWS EC2 instances.
Many enterprises are reluctant to allow direct code deployments to production -- not because they lack the technical ability, but because they believe it's a security risk not to enforce separation of duties through artificial gates between developers and production. However, IT experts increasingly see such gates as unnecessary, and view GitOps as a means to limit or eliminate human access to production, which they argue is a greater security risk than automated application deployments.
At Procore, GitOps has replaced hands-on SSH access to production and error-prone manual deployments. If errors or vulnerabilities are discovered post-deployment, the Procore IT team uses Capistrano's rollback features to correct them.
"[GitOps] also helps us with audit, to get eyes on [code] faster in case problems come up," Espinola said.
Procore baked in compliance and vulnerability scans on developers' code commits and third-party package dependencies to its CircleCI continuous integration tests, and embraced test-driven development to further secure its workloads. If all else fails, frequent minor releases limit potential damage, as opposed to large-scale application rollouts.
GitOps pipeline pre-empts Kubernetes
On the whole, enterprise IT is further along with Kubernetes and containers in production than with hands-free GitOps application deployments. For Procore, it's the other way around.
The company runs Docker Compose on EC2 instances and uses its CircleCI and Capistrano-based CI/CD tools to build and distribute images in production to hundreds of EC2 instances. Procore also uses Puppet's configuration management tool to configure and distribute Envoy sidecar proxies within that infrastructure to shape incoming Internet traffic within its infrastructure. Kubernetes and Istio, the tools many companies put into place to manage large-scale Docker and Envoy deployments, remain in the staging environment for now.
Our continual ability to deliver product, iterate and test through GitOps is what has allowed us to widen our [product line] to include building owners, architects and subcontractors.
Kyle EspinolaSenior software release engineer, Procore Technologies
"Container orchestration is our area of focus and direction for the future," said Espinola. "We're driving to a point where we've got the ability to bring up full environments using things like Kubernetes, with all the services required in production, to make sure everything's working end to end for our growing R&D departments."
That growth for the company has been fueled by the transition to GitOps and the IT team's focus on solving specific business problems, Espinola said.
"Our continual ability to deliver product, iterate and test through GitOps is what has allowed us to widen our [product line] to include building owners, architects and subcontractors," he said.
On a technical level, the business problem that launched Procore's GitOps journey two and a half years ago actually was very simple.
"It was impossible for any one developer to run our whole test suite on their laptop -- it would take around a day or so," Espinola said. "One of the first things that got us involved with [CircleCI] was their ability to run our test suite in parallel across multiple containers -- from Day One, that got us hooked."
Procore's plans include integrating QA engineers into the test-driven development process and using CircleCI's Orbs to integrate more third-party tools into its GitOps pipeline. The company is also testing CircleCI roadmap tools that monitor the pipeline for misconfigurations and performance or cost inefficiencies.
Dig Deeper on Systems automation and orchestration
