Use Jenkins for deployment, support and provisioning in ops

CI/CD basics

Successful software development consists of processes that bring consistency and speed to the resulting software product. Traditional waterfall development paradigms are slow and demand substantial investments to remediate. New development paradigms are far more iterative -- they break development efforts into smaller and readily accomplished tasks. The ever-growing emphasis on iteration enables developers to move through the coding, building, testing and delivery-deployment pipeline in a few hours. If an error appears or a result is undesirable, developers can repeat the iterative cycle to address the issues.

The cycle can be so fast and repetitive that the effort can seem continuous. Today, CI/CD pipelines and other DevOps processes are standard in software development. A development cycle can not only occur quickly, but also concurrently by different developers and teams.

But CI/CD paradigms pose challenges. A DevOps software project can involve numerous developers and teams, all working on different pieces of the project simultaneously. This presents daunting process and control issues that demand careful automation and orchestration along the process pipeline. Tools are key to any continuous process. While a CI/CD pipeline might involve dozens of different tools, a central element of the pipeline is an integration server.

What does Jenkins do?

Jenkins is a popular integration and automation server that oversees and orchestrates the principal steps of a CI/CD pipeline.

A typical CI/CD process involves coding, building, testing and deployment. With so many components and steps, it's easy to see how the pipeline can become complex:

1. Developers write and make changes to the codebase. They store the code in a repository, such as Git, which might involve a static code analysis to validate the code style and look for common vulnerabilities.
2. Then, developers combine the code changes with other code elements, such as libraries, and translate them into executable machine code through the build step.
3. Teams test the new build against established test criteria to validate the build's performance and look for bugs. Document any issues and loop them back to developers for remediation.
4. A successful test suite then enables the build to deploy to production or general availability.

A CI/CD pipeline involves lots of moving parts that are difficult for humans to manage. A tool such as Jenkins takes up a core role in the pipeline and brings order to the chaos. Consider how an automated integration server can manage the pipeline:

• When a developer commits a code change to the repository, the integration server recognizes the update and triggers a build. Build errors loop back to the developers.
• With the build now stored, the integration server launches a series of appropriate tests on the build. Any initial testing issues loop back to the developers.
• If the new build passes the initial testing, the integration server stages the build for more comprehensive testing against established test patterns and criteria. Testing issues loop back to the developers.
• If the build passes testing, the integration server can opt to deploy the build to production automatically or await human approval to execute the deployment.

How Jenkins works

Jenkins fundamentally operates in jobs. Each job involves a number of steps.

Each build project is a job, and Jenkins supports a range of common job types. In a freestyle job, admins combine any software configuration management and build system. An external job records processes running outside of Jenkins and displays that information on a Jenkins dashboard. Configuration variables pass to a parameterized Jenkins build at runtime. A multiconfiguration job helps in more sophisticated projects, such as platform-specific builds.

Jenkins creates an orchestration workflow, called a pipeline, for projects that don't fit easily into freestyle jobs. As an example, a declarative pipeline file includes a build stage, a test stage and a deploy stage; each stage involves many individual steps that do the actual work. Jenkins can create and support multibranch pipelines to orchestrate complex build jobs according to outside criteria. In a continuous or DevOps model, teams use Jenkins for deployment activities, even in complex setups and across multiple Jenkins servers.

Plugins integrate Jenkins with build tools, version control systems, test systems, bug databases and other diverse tools that enable software development and deployment. For example, with the Google Compute Engine plugin, Jenkins uses GCE VMs when executing build tasks, rather than having an admin provision and configure VMs. Plugins can perform unit tests and automated testing, and the test results are also supported by plugins that display report data within the Jenkins interface. There are many plugins from vendors and independent third-party developers, and this broad support has made Jenkins widely adopted among diverse development teams.

Jenkins admins rely on the dashboard to manage tasks, such as adding new jobs, managing the Jenkins server, and viewing jobs in the queue or currently executing. It displays a timestamped build history with pass/fail results.

Jenkins plugins

Plugins are the key to Jenkins' flexibility and extensibility. They enable direct interoperability with a wide range of build tools, test platforms, cloud providers for deployment and analytics tools across the CI/CD pipeline. Admins can download almost 2,000 plugins from the Jenkins Update Center. The plugins fall into five principal categories:

• Platforms
• User interface
• Administration
• Build management
• Source code management

Admins can install plugins through the Jenkins web UI or the Jenkins command-line interface. In addition, Jenkins administrators can update, disable, remove or uninstall plugins as required to maintain an up-to-date pipeline environment.

Jenkins provisioning, support in operations

Today's development pipeline doesn't stop at IT operations' door. Operations staff use Jenkins to deploy within a modern CI/CD pipeline and to automate routine application support functions.

Operations staff can rely on Jenkins for day-to-day tasks currently performed via ad hoc scripts or facilitated by other orchestration and automation tools. Jenkins combines with automation and orchestration tools to accelerate the execution of common tasks, bring consistency to them, and eliminate errors and oversights that can compromise security or performance.

In an operations setting, Jenkins can automate the routine maintenance tasks on crucial business applications, such as garbage collection, database compaction, file backups and disaster recovery setup. These tasks are often performed manually by IT staff. Jenkins can run the tasks and even configure a sequence of them into an orchestrated process.

Ops teams can use Jenkins in conjunction with other infrastructure tools, such as Microsoft System Center Configuration Manager (SCCM) -- renamed in version 1910 to Microsoft Endpoint Configuration Manager (ECM) -- to configure and deploy new servers in the infrastructure. IT staff connect the hardware to power and the LAN, then the tools take over: SCCM and ECM recognize the new hardware ID and trigger Jenkins to perform prescribed installations to prepare the system for production.

IT operations saves time and enforces consistency with Jenkins code delivery from dev, provisioning and support. It also enables ops and dev to speak the same language, understand each other's priorities and work better together, encouraging a DevOps culture.

Jenkins and containers

Containers are a vital type of virtualization, replacing traditional VM deployments with small, portable and resource-efficient virtualized containers. Docker is the oldest and most broadly accepted container engine available to create, deploy and manage containers. Jenkins supports Docker, and the two tools can work together for greater speed and consistency in automation.

For example, Jenkins can operate within a Docker container. But a system must have a Docker engine installed and running. An administrator can download and run Jenkins within a Docker container by simply executing a command line, such as the following:

docker run --name jenkinsci -p 8080:8080 jenkins/jenkins:lts

Once the Docker engine is installed, administrators can easily denote the automatically generated passcode and access the Jenkins installation with a localhost command, such as the following:

http://localhost:8080/

This launches Jenkins and enables administrators to configure Jenkins, install and manage plugins, create pipelines, and manage jobs.

The advantages of using Jenkins in a container are isolation and flexibility. Administrators can stop, move, restart and destroy containers. This enables them to clean up resources and free space on the server that might otherwise be committed to a bare-metal Jenkins installation.

Jenkins can also use Docker and containers as deployment targets for tested and approved builds rather than traditional VMs. Numerous Jenkins plugins support Docker environments, such as the following:

• Docker plugin for Jenkins
• Docker Pipeline plugin
• Docker Build Step plugin
• Kubernetes plugin

Where to get Jenkins

Jenkins is readily available for download through the official Jenkins website. Admins should check for the latest hardware requirements and can choose from a variety of downloads for select OSes, such as Ubuntu, Debian, CentOS, Fedora Red Hat, Windows, Docker, OpenSUSE, FreeBSD, Gentoo, macOS and OpenBSD. Jenkins is based on the MIT License and is freely distributed to use and modify. Some third parties maintain versions of Jenkins, which can offer some support.

Downloads are available as stable versions on 12-week update cycles, as well as weekly releases that include bug fixes and features, but might not be as stable. A weekly release is ideal for testing and evaluation, while a stable long-term support version is appropriate for use in production.