Netflix and Google's Spinnaker pipeline tool has improved its integration with Kubernetes, which should add to its appeal within enterprise DevOps shops where Spinnaker has accelerated continuous deployment.
Spinnaker pipelines are highly customizable, but default settings developed at Netflix can give less advanced DevOps shops a leg up as they implement continuous deployment for applications. Some users have replaced another open source pipeline tool, Jenkins, with Spinnaker because it was easier to use. However, previous integrations between Spinnaker and Kubernetes container clusters needed improvement, which was delivered this week.
Version two of Spinnaker's Kubernetes plugin, which is called a provider, integrates with the native kubectl command-line interface (CLI) instead of the previously used Java library, and adds support for custom resource definitions that facilitate management of third-party infrastructure components alongside Kubernetes.
This means that the v2 provider automatically retrieves Kubernetes manifests for the container images a user wants to deploy, and generates multiple versions of those manifests for staging, QA and production environments as part of the Spinnaker pipeline. IT ops admins can also use the new provider to restrict developers' access to Kubernetes clusters for security purposes.
"I'm not afraid of the Kubernetes provider anymore," said Gard Rimestad, staff DevOps engineer at Schibsted Media Group, a multinational media company in Norway, a V2 early adopter. "This one is very close to the Kubernetes code, and means we can use the same versions of Kubernetes with Spinnaker in the cloud that we use locally."
Google and Netflix also borrowed Kubernetes' early-stage governance structure for the Spinnaker project this week. Under the new governance plan, the project will get two steering committees, one to oversee the overall project roadmap and another to evaluate technical strategy, which will open up Spinnaker pipelines to more contributors over the next year. This is a prerequisite to Spinnaker's eventual donation to an established foundation, such as the Cloud Native Computing Foundation, its parent The Linux Foundation or another group, said Andy Glover, director of delivery engineering at Netflix.
Spinnaker pipelines help users get CD off the ground
Netflix created the Spinnaker pipelines tool in 2014 for internal use, and Google joined the project when the continuous deployment software was released to the open source community in 2015. Netflix has long been seen as an elite user of technology. But since June 2017, and the Spinnaker 1.0 release, the Netflix pipeline approach has caught the attention of more earthbound enterprises.
There's work in the Spinnaker community on a way to say, 'Deploy this thing, I don't care how.' ... You can seamlessly update the 'how' under the hood as technology changes.
Andrew Dobsonsoftware engineer, Vistaprint
"We made seven attempts at continuous deployment starting five years ago, and they all failed," said Brandon Leach, senior engineering manager for continuous deployment at Lookout, a mobile device security company in San Francisco. Lookout found uneasy equilibrium for a while with four different Jenkins pipelines, but still searched for a better approach, which it found in Spinnaker in 2018.
"Spinnaker let us orchestrate infrastructure from the perspective of a code release," Leach said. "We thought about how we wanted the release to happen, and then infrastructure just happened for us, in a much smoother way than blue/green deployments that used Terraform wrappers."
Vistaprint USA Inc., which custom-prints marketing materials for small businesses, replaced a continuous deployment system based on Atlassian's Bamboo and Jenkins with Spinnaker pipelines. Vistaprint looked to replace its previous pipelines after under-provisioned Bamboo builds took longer than a day to spin up, and production releases got stuck waiting on those tests to finish. The company tested and compared Spinnaker vs. Jenkins combined with application deployment code written in-house, and Spinnaker won based on its ease of use.
"I had a basic Spinnaker pipeline up and running in a day, and you could see it do [deployments] in another day and a half," said Andrew Dobson, software engineer at Vistaprint, in Lexington, Mass., a subsidiary of Cimpress, based in the Netherlands.
Spinnaker’s pipeline interface automates continuous deployment steps for DevOps shops.
Spinnaker pipelines can abstract nitty-gritty infrastructure configuration tasks away from DevOps teams, but users can also tinker with Spinnaker's code. Lookout's Leach wrote his own infrastructure provider for Spinnaker to support a container orchestration environment on Amazon ECS instead of Kubernetes. Schibsted's Rimestad blended Spinnaker pipeline tools with his company's own open source deployment daemon.
Spinnaker pipeline users draw up wish list
Spinnaker pipelines help enterprises navigate the early stages of continuous deployment, but as users grow more sophisticated they look for more features, such as automated evaluation, promotion and rollback of software artifacts in the Spinnaker pipeline before the continuous integration stage. Users also said the tool could improve its API support for stateful app deployment, and stabilize and simplify the CLI for Spinnaker pipelines. Many are eager to get their hands on declarative pipeline tools still at the alpha stage within Netflix.
"There's work in the Spinnaker community on a way to say, 'Deploy this thing, I don't care how,'" Vistaprint's Dobson said. "That means you can seamlessly update the 'how' under the hood as technology changes."
The Spinnaker pipeline roadmap also includes the artifact promotion features, enhancements to make the alpha version of the CLI easier to use, and updates for stateful application deployment APIs requested by users, the project's maintainers said.
"All stateful services at Netflix use Spinnaker's APIs directly to affect delivery," Netflix's Glover said. "Cassandra, our encoding farms, and our Kafka messaging services have built some custom orchestrated deployments using Spinnaker APIs, but not pipelines, out of the box, [and] eventually we will build some of this functionality into Spinnaker."
Dig Deeper on Systems automation and orchestration
