Use Dapr on Kubernetes to build microservices in production

Make microservices agnostic

Distributed Application Runtime, or Dapr, was created at Microsoft in 2019 and later added to CNCF as an incubating project in 2021. The project aims to deliver foundational building blocks for microservices development to enable developers to focus on the business logic for a microservice without having to worry about other areas in the microservices landscape, including service discovery and state management.

Think of the building blocks, shown in Figure 1, as standard HTTP and gRPC APIs that Dapr exposes for common capabilities like service invocation, state management, and publish and subscribe messaging.

To solve crosscutting concerns for their apps, developers can make an HTTP and gRPC API call to a Dapr building block. For example, to use Dapr to store state for microservices, use an HTTP POST call to the endpoint http://localhost:<DAPR_HTTP_PORT>/v1.0/statestore with the payload.

Building blocks are blueprints, or interfaces, for the API. The actual implementations are called components. Think of a Dapr component as the code that implements the standard API for a building block with infrastructure components, such as Redis for state management.

Sidecar pattern

The sidecar pattern, also known as the sidekick pattern, is a popular microservices decomposition pattern. With a sidecar pattern, the crosscutting concerns of building a microservice, such as logging and monitoring, are implemented and run as separate services, as seen in Figure 2.

Implementing the sidecar pattern in Kubernetes requires understanding what a pod is. A pod is the atomic unit of deployment and typically contains a single container, but it can also house multiple containers that share the same network and volume.

A pod contains the container running the microservices' core logic, while a sidecar runs a separate service to provide the main application's extra features. The sidecar shares the same lifecycle as the parent container.

Dapr uses this pattern to deploy applications on Kubernetes; it deploys a sidecar, which provides the basic APIs -- building blocks -- the application needs. The infrastructure thus requires no code changes to enable Dapr integration. Underlying infrastructure implementations can swap in and out, as the building blocks ensure use via common and standardized API endpoints.

Try these tutorials in the Dapr docs for more clarity on the developer workflow involved in building applications with Dapr.

Deploy a Dapr application to Kubernetes

Dapr docs tutorials deal mostly with local app development. Running Dapr apps in standalone mode is not advised in production environments. It's possible to run Dapr apps in a Kubernetes cluster or a serverless offering, such as Azure Container Apps.

This tutorial will discuss Kubernetes cluster configurations. For the rest of the setup, I have a Kubernetes cluster up and running using kind, a tool to run local Kubernetes clusters. In addition, the Dapr control plane is already set up using the Dapr command-line interface (CLI) via the dapr init -k command.

Deploying a microservice that uses Dapr on Kubernetes at a high level is a two-step process:

1. Set up the Kubernetes cluster with Dapr components.
2. Configure the application deployment manifests.

Set up the Kubernetes cluster

To support applications that require Dapr sidecars, you must configure the underlying Kubernetes cluster beforehand. Below are some of the recommendations when deploying Dapr for production.

Dapr deployment

One installation option is to use the Dapr CLI and Helm charts. It is recommended to use Helm version 3 to install Dapr on a Kubernetes cluster. When using Helm to deploy Dapr, create a values file and check that into a version control to track changes. Use GitOps with Helm to install and upgrade changes to the Dapr control plane. Deploying Dapr via Helm also provides zero-hour upgrades to the Dapr control plane and sidecars.

Dapr configuration

To use Dapr in a production environment, deploy it in a high availability (HA) mode for the Dapr control plane. This sets the replica count to three for each control plane pod in the dapr-system namespace. HA mode enables the Dapr control plane to survive node failures and other outages.

For a new Dapr deployment, the HA mode can be set with both the Dapr CLI and with Helm charts. Install Dapr control plane components in a dedicated namespace. By default, the CLI and Helm charts use the dapr-system namespace. Dapr supports scoping components and service invocation considerations. Namespaces can limit component access to particular Dapr instances.

If you've already invested in service mesh implementation, then Dapr can work well with other service meshes.

Perform capacity requirements and resource planning

The cluster itself should run at least three worker nodes to support the highly available control plane installation. If one is running on top of a cloud-managed Kubernetes offering, run it in the HA configuration recommended by the managed offering for the cluster itself.

A Kubernetes workload best practice is to set resource requirements for the Dapr control plane components deployed to the Kubernetes cluster. Start with the recommended capacity and tweak as per the needs of the team or organization. Set the resource requirements for the instances of the Dapr sidecars deployed with an application as well.

Security configuration recommendations

Use a private container registry to host application container images. For example, Azure Container Registry is a good option for an Azure Kubernetes cluster as it can be granted permission to pull images using managed identities, and both components can be made private without a public endpoint.

When configured properly, Dapr ensures secure communication. It also has a number of built-in features to increase application security.

For example, though Dapr has mutual authentication -- mutual Transport Layer Security, or mTLS -- active by default, ensure that it is enabled to secure communication between the Dapr sidecar instances. Users can bring in their own certificates. Enable bidirectional authentication for communication between the application and Dapr API. Adding authentication for this communication adds another layer of security.

Dapr component manifest files -- written in YAML -- should have secret data configured in a secret store, not hardcoded in the file. Install the Dapr control plane on a dedicated namespace, which is set on dapr-system by default.

Dapr supports scoping components for certain applications by using Kubernetes namespaces. If that level of isolation is required, enable those requisite scoping components.

Monitor configuration

Dapr's default settings enable tracing and metrics, but users should also set up distributed metrics and tracing for the Dapr control plane, as well as the applications deployed to production. Users can deploy an instance of Zipkin to forward these Dapr apps and control plane traces.

Dapr defaults expose a Prometheus endpoint on port 9090, which Prometheus can scrape for data.

Below are the default components deployed in the dapr-system namespace.

• dapr-operator. Manages the Dapr component's lifecycle and service endpoints.
• dapr-sidecar-injector. Injects Dapr sidecars into the deployments with specific annotations and adds the environment variables specific to Dapr -- for example, DAPR_HTTP_PORT and DAPR_GRPC_PORT.
• dapr-sentry. Runs as a certificate authority and manages certificates required to enable mTLS for communication between Dapr sidecar instances.

• dapr-placement. Required only if the application uses actors with Dapr.

Configure the application and microservice

Once the cluster is set up with the Dapr control plane, deploy the microservice application via Dapr.

For the purpose of this tutorial, refer to this GitHub repository for the python-dapr-demo, which uses the Dapr state management building block and Redis, deployed on Kubernetes, as the underlying Dapr component. This component is deployed using the Dapr quickstart tutorial for convenience; in a production environment, deploy and manage these state stores in the cloud or data center.

The code repository layout is straightforward, as shown in Figure 3.

The GitHub actions pipeline is running continuous integration, and building and publishing the container image to Docker Hub.

In the above GitHub repository, the Kubernetes manifests/redis-state.yaml file defines the state management component using Redis deployed on the Kubernetes cluster itself.

apiVersion: dapr.io/v1alpha1 
kind: Component 
metadata: 
 name: statestore 
 namespace: default 
spec: 
 type: state.redis 
 version: v1 
 metadata: 
 - name: redisHost 
   value: redis-master.default.svc.cluster.local:6379 
 - name: redisPassword 
   secretKeyRef: 
     name: redis 
     key: redis-password

The application itself is deployed into the Kubernetes cluster using the manifests/app.yaml file. For the Dapr control plane to inject sidecars, it must be annotated with the specific tags.

Below is the list of annotations placed on the deployment, which deploys the sample Python app.

annotations: 
       dapr.io/enabled: "true" 
       dapr.io/app-id: "pythondaprdemo" 
       dapr.io/app-port: "8080"

Once these annotations are placed and the manifests are applied, use the command below.

> kubectl apply -f ./manifests

The Dapr control plane injects the Dapr sidecars inside our application deployment. Also, it injects environment variables inside the application container.

Let's see the pod we created after we applied the manifests on the cluster. In this tutorial, we use Lens IDE to show the pod for the python-dapr-demo app.

Figure 4 depicts the pod running the application code, which uses Dapr for state management. It also displays how the Dapr sidecar container is injected along with environment variables.