How to use Azure Arc-enabled Kubernetes, step by step

Azure Arc-enabled Kubernetes

ARM enables the end user to control and manage resources; Azure Arc extends that control plane by enabling users to add external resources to the Azure environment. Resources added via Azure Arc are represented as 'first-class citizens' inside Azure. They have Resource IDs and access to other out-of-the-box Azure governance features, such as Azure Policy, and an entire observability stack. This stack includes Azure Monitor infrastructure and application logs, metrics, events and traces. Read more about Azure Arc-enabled Kubernetes on official Microsoft documentation.

On top of Azure policies and built-in monitoring benefits, IT organizations can use GitOps to configure Azure Arc-enabled Kubernetes clusters, which pushes uniform configurations to all Kubernetes clusters across the control plane. Managing these clusters in multi-cloud or vendor environments can be time consuming, costly and confusing. But Azure Arc and GitOps simplify and streamline these configuration deployments via improved controls.

What is GitOps?

GitOps is a combination of Git and Kubernetes to model infrastructure as code (IaC) and continuous integration and delivery pipelines (CI/CD) for declarative software application and infrastructure deployments.

In traditional CI/CD systems, once the CI is triggered, some configurations are pushed forward in the pipeline. But GitOps continuously monitors any changes to a Git repository -- and the pull configuration changes from this repository. This repository acts as single source of truth for declarative infrastructure and application deployment. Azure Arc uses Flux under the hood to monitor these configuration updates, deployments and the desired state of Kubernetes cluster.

How do Azure Arc and GitOps work?

On a high level, once an external Kubernetes cluster connects to Azure Arc, the following steps are performed:

1. Azure Arc installs metrics and Flux agents on the local cluster.
2. Arc sets up a managed identity on the Kubernetes cluster automatically; users can also choose a service principal, but managed identities ease their management demands.
3. After successful setup, the agent begins to send metadata from the cluster to Azure.

If a user creates a Kubernetes GitOps configuration on Azure Arc-enabled Kubernetes, then the following steps are performed as well:

4. A Flux agent monitors the Git repository URL for IaC manifest files.
5. If a change is detected, Flux pulls the manifest files to a local repository.
6. Flux configures the local Kubernetes cluster to adapt the configuration changes.

Before we look into each of these steps in more detail, let's first set up our cluster and environment.

Create the environment

For this tutorial, I'm using Minikube to run a Kubernetes cluster on my personal desktop, which runs on Windows 10 OS. And I'll walk you step-by-step through onboarding your first Kubernetes cluster to Azure Arc.

The end goal is to demonstrate that any Kubernetes cluster can be onboarded easily to the Azure control plane as Azure Arc-enabled Kubernetes -- irrespective of where it runs, whether that's on on-premises servers, a desktop or any of the major cloud providers like GKE or Amazon EKS.

Prerequisites

• a Microsoft Azure account and access to the Azure portal;
• the latest stable release of the Azure command-line interface (CLI) to manage Azure resources;
• the latest stable release of Helm Charts, a package manager for Kubernetes;
• a running Kubernetes cluster to which we can connect and configure on Azure Arc; and
• public Git repository with Kubernetes deployment manifests; this tutorial uses a basic Nginx deployment manifest.

Connecting local Kubernetes cluster to Azure Arc

To get started with Kubernetes development, you can use Minikube and, for simplicity's sake, this tutorial runs the Kubernetes cluster on Minikube.

Minikube is a lightweight Kubernetes implementation that creates and launches a VM on your local system and deploys a single-node Kubernetes cluster for daily development needs.

The first step is to ensure is that the cluster is up and running. So let's start Minikube from any CLI -- this tutorial uses the PowerShell console -- with admin-level privileges.

minikube start

Before we can onboard our Azure Arc-enabled Kubernetes cluster, we need to create a home for the cluster -- by which I mean a new Azure resource group. Let's log into our Azure console via the Azure CLI:

az login

Once you log into Azure through the CLI, the console will display subscription information, as shown in Figure 3.

Now let's create a new resource group named first-azure-arc-k8s in region westus2 with the following command. This enables us to connect our cluster to this resource group later on in the tutorial.

az group create --name first-azure-arc-k8s --location westus2

With the resource group created, we must ensure that Azure CLI extension connectedk8s is installed; this is a requirement to get the az CLI commands to connect our cluster to Azure Arc.

Check if the extension is installed:

az extension show --name connectedk8s

Install the extension:

az extension add --name connectedk8s

After we install the extension, run the following command to connect the local Kubernetes cluster to Azure as an Azure resource -- minikube-k8s -- in the resource group we created.

az connectedk8s connect --name minikube-k8s --resource-group first-azure-arc-k8s

You can verify this connection in the Azure portal where a new Azure Arc-enabled Kubernetes resource is created in the resource group, as shown in Figure 6.

To establish this connection, a few deployments and agents are configured on the local cluster in the azure-arc namespace. You can verify from Figure 7, for example, that there is a Flux agent running to pull declarative IaC and support GitOps. We can also verify there is an Azure Monitor agent to collect monitoring data for Azure Monitor. To check these Azure Arc deployments and pods on the local cluster, run the command:

kubectl -n azure-arc get deployments,pods

Configuring Azure Arc-enabled clusters with GitOps

There are two methods to set up GitOps on an Azure Arc-enabled cluster. The first option is to use the Azure portal via the Kubernetes resource we connected, and then navigate to Settings > GitOps > Add Configuration.

The second approach is through the Azure CLI. So to configure GitOps, we install an Az CLI extension named k8sconfiguration.

az extension show --name k8sconfiguration
az extension add --name k8sconfiguration

Once the extension is installed, configure the local Kubernetes cluster with GitOps using a public Git repository hosted on GitHub.

Ensure you supply the correct cluster and resource group name to the command.

 
az k8sconfiguration create \
    --name azure-arc-config \
    --cluster-name minikube-k8s \
    --resource-group first-azure-arc-k8s \
    --repository-url https://github.com/PrateekKumarSingh/azure-arc-demo \
    --scope cluster \
    --cluster-type connectedClusters

The repository mentioned in the above command contains a manifest file deployment.yaml to deploy an Nginx server on a Kubernetes cluster. We configure this manifest in GitOps in our Azure Arc-enabled Kubernetes cluster so that it deploys automatically to our local Kubernetes cluster. The Flux agent can monitor this repository for changes and adapt the cluster for any changes in the file.

By default, this GitOps configuration will monitor the 'master' branch on the public repository URL you supply. However, newer repositories have a root branch named 'main' and, because of this, your deployments might not trigger.

If your public repository has root branch named 'main,' make sure you supply the name of git branch using --operator-params='--git-branch=main' to the 'az k8sconfiguration create' command mentioned in the previous step. Read more about this parameter in the Azure documentation.

With our configuration created and installed, we can verify that new Nginx pods are deployed on your local Kubernetes cluster.

This is the same configuration in the manifest file hosted on the public GitHub repository. The configurations in Figure 11 verify that the Kubernetes namespace -- along with the deployment with two replicas and service -- are mentioned in the manifest file.

GitOps in action

As soon as GitOps is configured with an Azure Arc-enabled Kubernetes cluster, it will pull the configurations from the public Git repository. But can GitOps reconfigure deployments in the local Kubernetes clusters for changes to the manifest file in the repository?

Let's check it out.

First, we'll clone the GitHub repository locally and edit the YAML file to change the number of replicas deployed from two to three.

git clone https://github.com/PrateekKumarSingh/azure-arc-demo/
cat .\azure-arc-demo\nginx\deployment.yaml | grep replica 
notepad .\azure-arc-demo\nginx\deployment.yaml
cat .\azure-arc-demo\nginx\deployment.yaml | grep replica

Once changes are done locally, we commit those changes through the git commit command and push the changes upstream to the public repository.

cd .\azure-arc-demo\
git add .
git commit -m "changing replica set to 3"
git push origin master

Within a minute of pushing this change, the Flux agent running in the local Kubernetes cluster will pick these configuration changes from the repository and begin another replica deployment. This replica deployment is another Nginx pod in our cluster, which we can verify from the cluster as shown in Figure 12.

kubectl get pods -n nginx-webserver -w