Get the most out of continuous testing with Jenkins

Implement CI/CD with Jenkins

CI/CD are offshoots of the Agile development methodology. CI continuously gives feedback, whereas the execution of CD is more or less scheduled for the end of each Agile iteration. Any change that is released into the production environment must be somewhat coordinated across the entire software development and DevOps teams.

CI is the process of compiling one's larger project, which entails individual programmers checking their code into a code repository, such as GitHub or Subversion. Accordingly, CD is the practice that pushes that code out to its target environments. CD includes infrastructure steps, like creating virtual machines and containers, deploying to Maven repositories, as well as configuring networks and storage. New and updated code moves through these various steps along a CI/CD pipeline orchestrated by the automation server, such as Jenkins.

At each step of CI/CD, development and test teams can run tests. The assessments generally fall into the following categories:

• Functional: Functional tests check that the whole program does what it is supposed to do from a functional point of view.
• Unit: Unit tests check that individual subroutines in a program work.
• Smoke: Smoke tests check the most important functions before pushing onto the further tests.
• Integration: Integration tests check that various systems connect with each other.
• Load: Load tests verify software performance under the requirements demands.

When the code passes the tests, Jenkins goes to the next step. When it doesn't, Jenkins can be made to stop. However, these assessments don't always provide Jenkins with the most comprehensive feedback loop. So, the software tool might know that a test ran, but it might not really know whether it ran successfully.

Select quality plug-ins to improve CI/CD. Specifically, choose plug-ins that push the code toward the target environment and enable event-driven feedback into the push process.

When Jenkins' dashboard lights up red, that indicates failure, and developers and testers can look at the console's output or Slack, if your team utilizes it. Slack is a messaging tool that lets teams chat with each other and pull in other resources, like code deployment and analytics data. It is often used as a place to report Jenkins build output. Slack keeps output in neat channels to avoid overwhelming the conversation with too much clutter.

Still, the output from these builds is often too verbose and large to understand at a glance. Tools like Sauce Labs, Postman and BlazeMeter make CI/CD tests easier to understand.

Continuous testing with Jenkins within Java

For programs written in Java, Jenkins typically runs through these steps:

1. Developers create a phase in Jenkins that runs Apache Maven or Gradle to compile their code. These packages define dependencies, meaning external JAR files, that a program needs to include.
2. Programmers then check their code into the code repository.
3. A check runs that will let the programmers know whether the subroutines they have written will clash with what other developers wrote.
4. Programmers also execute test code, written with frameworks like JUnit, to test individual subroutines. Meanwhile, functional tests check the whole program. This second approach is often called business-driven development and is made easier with natural language frameworks, like Cucumber.

To illustrate, imagine a unit test as a math equation. Suppose we have a function square (x) that calculates the square of x:

square (x) = x * x

That is in one program. The programmer also writes a second program that, in this case, says:

assert (square(2)) = 4

The second program tests whether or not the subroutines correctly calculate the square of two. If the result is not four, it throws an assertion error, and Jenkins will stop. In practice, natural language testing works more like:

if bank balance < $100 and withdrawal is > $100

This process is typically how a framework like Cucumber works. This particular test automation tool is used in conjunction with business-driven development, an approach that has you write the test code first. With no logic written yet, Cucumber will fail on the first try. At this point, you would need to write the minimum amount of code until the test passes. So, the focus is to get the bare-bones pieces of the program to work upfront. This avoids wasting time on tasks that are not on the critical path.

Continuous testing can also test more than business logic or subroutines. The webpage and mobile app can also be tested using tools like Selenium. Testing tools emulate various browsers and browser versions with different mobile devices. Such programs can test, for example, when users click on the next button, the next screen comes up and not something unexpected.

The final step of continuous Jenkins testing is to assess that CD works. This is often implemented using SaltStack, Ansible or other build and orchestration tools.