denisismagilov - Fotolia
Three DevOps toolchain options refine IT automation
DevOps tools that don't interconnect form a bottleneck that teams can easily avoid -- or retroactively remedy -- if the CI/CD workflow merits the time commitment.
DevOps toolchains are becoming more integrated, from dev to test to IT.
Properly instrumented process automation can improve application quality and security -- a major DevOps goal -- through detailed performance profiles and code and configuration audits. DevOps automation reduces and eliminates rework, reuses code and configuration templates, and eliminates manual tasks.
As DevOps teams mature, they shift attention from automating individual steps and tool mastery to tying those automation tools together. As development and IT teams work more closely together, they naturally want to connect each department's tools to streamline the workflow. There's no lack of tools, but the amount of customization effort required to use and integrate them poses a problem. DevOps tools were designed to be eminently programmable, but not necessarily to work together.
The Holy Grail is an integrated DevOps toolchain that spans process silos and links up each stage of the development, integration and delivery process. DevOps teams have options when they create toolchains, ranging from a fairly cohesive setup based on application programming interfaces (APIs), to one that takes considerable DIY integration work. Generally, the DevOps toolchain encompasses four categories:
- Build automation and source code management, such as Git and GitHub, Bitbucket or similar cloud services;
- Continuous integration software, such as Jenkins or Travis CI;
- Application deployment and configuration management tools, including Ansible, Chef and Puppet; and
- Infrastructure deployment software, such as HashiCorp Terraform or Amazon Web Services (AWS) CloudFormation.
Whether a DevOps toolchain integration is worth the time to stitch together depends on the team's continuous integration and continuous delivery (CI/CD) process and how often applications flow through the chain.
A common goal for DevOps tool vendors
There's hope for a cleaner, more consistent solution to the DevOps toolchain puzzle in the form of universally adopted standards. The DevOps Express initiative launched in 2016 to define reference architectures, best practices and integrations that can collectively accelerate DevOps adoption. The effort aims for technologies and reference designs that span multiple components across the DevOps lifecycle.
Although the organization has an impressive list of members, including Atlassian, CA, Chef, CloudBees, GitHub and Puppet, so far, it doesn't offer much beyond a bare-bones website. As standards are sorely needed, DevOps practitioners are encouraged to follow, and perhaps contribute to, this group.
The most popular tools are open source or include open APIs, meaning that, with a little effort, an intrepid team can make almost any DevOps software talk.
The infrastructure used for development and delivery is the paramount factor that determines a toolchain integration approach. The particulars for how tools in each category share information and directions vary.
The all-cloud DevOps toolchain
Organizations that most aggressively adopt cloud services are in luck because the large public cloud providers offer various services at each stage of the toolchain with APIs, sample code and reference designs detailing how to integrate them. Cloud providers grew in part thanks to this target user base that runs Agile, CI/CD processes for rapid application development and deployment.
Fervent public cloud adopters may rely on shared services for their entire DevOps toolchain. For example, AWS has four key services: CodeBuild for source and build management, CodePipeline for continuous integration and testing, CodeDeploy for deployment and CloudFormation for infrastructure automation that can be used to build a CI/CD pipeline without any manual coding. The process uses CloudFormation to set up and configure infrastructure and application stacks, and is mostly controlled via the AWS graphical user interface. The pipeline pulls source code from GitHub, AWS' equivalent service CodeCommit, or Amazon Simple Storage Service; builds and tests deployments using TeamCity, which automatically spins up to Amazon Elastic Compute Cloud (EC2) as part of the pipeline; and deploys the finished application to EC2.
Public cloud automation options
DevOps teams can create an integrated toolchain using the respective developer services from their provider:
- AWS CodeDeploy
- Azure Automation Desired State Configuration
- Google Cloud Deployment Manager
Microsoft Azure CI/CD tooling uses Visual Studio Team Services to automate the development side of DevOps and combines various Azure features, notably resource templates and desired state configuration, to control the operations side of application deployments.
Other cloud providers offer standardized ways to integrate with popular CI/CD tools. For example, IBM Bluemix offers templates for connecting to Travis CI.
DIY or buy DevOps toolchain approaches
Building DevOps toolchains out of local servers and software typically requires more do-it-yourself integration work, particularly with popular open source tools. Integration details are highly dependent on the tools. For example, a typical developer chain of Git for source code and build management, Jenkins for CI, and Chef for configuration management and deployment can be stitched together using ready-made modules due to each tool's built-in extensibility and APIs.
Custom configuration code should be minimal if you have extensible tools with plug-ins. The Jenkins Git module only requires the user name and email for the primary Git account and URLs and credentials for each code repository. Likewise, a Puppet plug-in, which notifies Jenkins when code deploys, requires that the user export a Puppet YAML report via HTTP by configuring the appropriate user ID and API token to point to the Jenkins server.
Users unwilling or unable to roll their own DevOps toolchain can choose from several packaged suites, including CA Technologies' Release Automation, IBM UrbanCode and Micro Focus Serena. The advantage to buying is that the vendor does all the integration. The downsides are cost, particularly for smaller organizations on a tight budget, and a set approach to the application lifecycle and DevOps processes that might not match your needs or existing process flow. Each organization must decide whether it provides a profitable ROI. There are trade-offs, and always this simple truth: When you must do a task often enough, even a long time spent automating it yields a positive return on the investment.
