An introduction to DevOps on AWS

Continuous integration and continuous delivery (CI/CD)

Continuous integration is a development practice to push and merge code changes into a centralized code repository. These are integrated continuously to detect changes and run a predefined set of automated actions, such as tests, scans and builds.

AWS offers several tools to help with the practice:

• AWS CodeCommit. A source control service on AWS that hosts private Git repositories and works seamlessly with existing Git-based tools. Because it is available on AWS portal as a hosted platform, developers can also use the console version to collaborate and commit, branch and perform other operations on the code repositories from a web UI.
• AWS CodeBuild. A continuous integration service that builds and compiles source code, runs a predefined set of tests and generates artifacts as deployable software.
• AWS CodeArtifact. An artifact repository service that enables DevOps teams to store, publish and share software packages built during various DevOps stages.

Continuous delivery, as the name suggests, focuses primarily on deploying code or configuration changes automatically when a change is detected in a source code repository. This ensures all changes developers push -- regardless of how small -- are deployed, tested and validated immediately. Because the small changes are deployed continuously in case of any issues or bugs, it is easier to detect the bug and revert back those changes. AWS DevOps tools that support this process include:

• AWS CodeDeploy. A deployment service that automates software packaging and deployments to various AWS providers and services, such as AWS Simple Storage Service (S3), Elastic Compute Cloud (EC2), Lambda or any custom service or location, such as on-premises instances. It eliminates manual deployments to ease deployments of software packages to a chosen instance and release features quickly with minimal or no downtime.
• AWS CodePipeline. A continuous delivery service that enables IT teams to define the steps and workflows required to release software. For example, deploy software packages to dev and staging environments first, where a series of tests and scans are run, before it's released to production instances.
• AWS CodeStar. A service that further speeds up the development process and overall experience with CI/CD tools in AWS. It creates a unified CI/CD interface that enables developers to set up the entire continuous delivery toolchain within minutes.

Infrastructure as code

Infrastructure as code is one of the most central principles of DevOps, where infrastructure is treated as software, including configurations and templates. IaC enables developers to rapidly provision, configure and manage AWS infrastructure in a consistent and secure way. This method replaces manual deployments and configurations, which not only saves time, but incorporates infrastructure as part of the application build and deployment.

CodeCommit stores infrastructure as code around which DevOps teams build pipelines to provision infrastructure via CodePipeline. AWS offers the following services to define infrastructure as a code:

• AWS CloudFormation. A service that enables developers to provision AWS resources and infrastructure in a consistent and predictable manner using an array of ready-to-use templates or by writing custom templates in specific syntax and structure.
• AWS Cloud Development Kit (CDK). A development framework that enables developers to provision infrastructure using programming languages like Python, .Net and Java, without leaving the comfort of an integrated development environment and rich text editor features like autocomplete.
• AWS OpsWorks. A configuration management service that helps developers automate server configurations on EC2 instances or on-premises servers, by providing managed Chef and Puppet instances.
• AWS Systems Manager. A service that helps automatically collect software inventory, apply OS patches, create system images and configure Windows and Linux OSes. Systems Manager is an end-to-end management tool that improves visibility and control into systems against any configurations drifts or noncompliance.
• AWS Config. A policy-as-code tool that enables more secure hybrid cloud environments by continuously evaluating configurations and activity against a set of predefined policies managed by security administrators.

Monitoring and logging

Observability into infrastructure health and application performance to detect failures, outages and performance bottlenecks is critical for every organization. To ensure everything is running at desired levels and state, AWS provides two different services for monitoring and logging.

• Amazon CloudWatch. A monitoring and observability service that automatically collects data and metrics from most of AWS' services, like EC2, S3 and Amazon Relational Database Service instances. Developers can also define custom metrics to be collected from any AWS resource. CloudWatch enables users to set up alarms, as well as auto scale and run an AWS Lambda function if an alarm is triggered.
• AWS X-Ray. A service that enables developers to analyze distributed traces in distributed applications -- like microservices -- to understand and debug any underlying performance issues or general failures.

Popular third-party DevOps tools

A single cloud provider cannot always provide the best tools for a domain as vast as a DevOps ecosystem, which AWS acknowledges via integrations with most third-party tools to give organizations the flexibility to choose what arrangement of tools, services and platforms works for them.

The following are some third-party tools that work well with AWS, broken into various DevOps categories:

• Continuous integration. For code repositories: Bitbucket, GitHub and GitHub Enterprise. For testing: Gremlin, BlazeMeter and Selenium for web testing.
• Continuous deployment. For building: Jenkins and JetBrains TeamCity.
• Infrastructure as code. Terraform and Pulumi.
• Monitoring and logging. Datadog, Prometheus, Grafana and Splunk Observability Cloud.
• Configuration management. Puppet, Chef and Ansible.
• Containerization. Docker, Docker Swarm and Kubernetes.

How to create a CI/CD pipeline in AWS

To understand what we have learned so far, let's create a simple CI/CD pipeline in AWS. In this AWS DevOps tutorial, we will:

1. Set up a code repository on CodeCommit with a static HTML file.
2. Create an S3 bucket to host a static website using the HTML file.
3. Set up CodePipeline, which is continuously integrated to CodeCommit -- our code repository -- to continuously deploy any new changes in the HTML file to the S3 instance.
4. Test the pipeline.

This pipeline will be triggered each time there is a new commit record in the CodeCommit repository. Once deployment completes, the static website will display any changes.

Create a repository on CodeCommit

Navigate to CodeCommit in the AWS console.

Under Repositories, click Create repository. Set the name of the repository -- for example, pipeline_demo -- and click Create. Once created, save the following HTML content to a file named index.html:

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
</head>
<body>
 <div align="center">
  <h1>Hello World!</h1>
 </div>
</body>
</html>

Navigate to Add Files > Upload to add this index.html file to the repository.

Scroll to the bottom of the page and click Add file > Upload file. And finally, commit changes to the 'main' branch.

Create an S3 bucket to store HTML for a static website

To get started with this next section, go to S3 service in the console. Create an S3 bucket with a unique name, such as "aws-devop-static-site" and take the following steps to continue setup:

1. Select the appropriate region.
2. Uncheck the Block all public access box.
3. Check the I acknowledge that the current settings might result in this bucket and the objects within becoming public box.
4. Click Create bucket, which will create a public S3 bucket.
5. Go to the bucket's "Properties" tab and scroll down to Static website hosting > Edit; click Enable.
6. Provide the value "html" in the index document box.

Save these changes before continuing on.

7. In the Permissions tab, scroll down to Bucket policy > Edit.
8. Copy the bucket's Amazon Resource Name (ARN) number and click Policy Generator. Within this form, perform the following two tasks:
   • For the Select Type of Policy drop-down, choose S3 Bucket Policy.
   • Add Statement(s), "Allow" Effect.
9. Set Principal to * and choose Amazon S3 as the AWS service.
10. Under the Actions drop-down, choose GetObject. Paste in this ARN: <S3_BUCKET_ARN_NUMBER_WE_COPIED>/*
11. Click Add Statement.
12. Click Generate Policy > and copy the generated policy output. Paste that policy text into the S3 Bucket policy editor.

Once again, save these changes before moving onto the next stage.

Create a pipeline on CodePipeline to push changes to S3 to serve as static website

1. Navigate to CodePipeline in the console.
2. Under Pipelines, click Create pipeline.
3. Provide a pipeline name, such as "static-website-pipeline" and click Next.
4. Under the Source page, select AWS CodeCommit as source provider, and choose the repository name we created in first subsection, which is pipeline_demo, from the drop-down and main as the name of the branch.
5. In the Change detection options, choose AWS CodePipeline and click Next.
6. Click on Skip build stage > Skip.
7. On Add deploy stage, choose Amazon S3 as the deployment provider.
8. Select the correct region and the name of the bucket created in previous sections.
9. Check the box Extract file before deploy, then click Next, and finally click Create Pipeline.
10. After a few minutes, your pipeline will be triggered. You can now see the S3-hosted static website on the public URL.

Test the AWS CI/CD Pipeline

1. Let's go back to our code repository on CodeCommit, make some changes to the html file and commit the new changes to the repository.
2. AWS CodePipeline will detect the change on the source repository and trigger a deployment stage, as shown in Figure 2.

3. This will publish your updated html file to S3, as shown in Figure 3.

4. Changes in the html file will be reflected on the static website, as confirmed by Figure 4.