Unpacking the event-driven microservices approach

There are many reasons why development teams might want to componentize their applications. Componentization facilitates reuse of common logic elements and improves both development efficiency and change management procedures. In operations terms, it makes applications resilient and scalable. Componentization is beneficial on both sides, so breaking an app into smaller components via microservices makes sense. This is why microservices development is such a hot topic and why event-driven microservices can potentially be very helpful for your application.

An event-driven application is one that responds in real time to real-world conditions reported by sensors or other application components. The structure of these applications varies depending on the specific mission, but there tend to be two major technological approaches. The first is complex event processing (CEP), in which correlation tools are used in front of the application to contextualize the events into meaningful conditions. The second approach is where the application itself contextualizes the events. Both approaches support microservices differently.

Events that have been contextualized by CEP tools, like open source Apache Flink, are essentially identical to business transactions. Applications that process these transactions follow an orderly workflow, and microservices are orchestrated along this flow based on the specific steps needed to process the transactions.

When you use microservices with CEP outputs, you should think in terms of API managers, brokers and service buses. In this situation, microservices are invoked like service-oriented architecture or REST components in order for the developers to know the message formats needed, the nature of stateless or stateful processing and the way that microservices are sequenced along the workflows. Traditional programming practices that optimize the reuse of the smaller components can be effective here.

Component reuse is an explicit benefit for microservices, and it has to be addressed in your design. Component reuse is facilitated by the fact that the sizing, in the functional terms of a microservice, used behind a CEP front end is almost totally controlled by the development team. Microservices with larger functional scopes aren't easily reused but are more efficient because there are fewer network paths to transit.

Noncontextualized events are usually processed by microservices, lambdas or functional components. They are then orchestrated in an orderly way by a separate component; this is referred to as orchestration, workflow engine or step function (often associated with AWS Step Functions). Event contextualization via microservices involves building a microservices-specific state machine, where each stage of processing is represented by a logical state and by work transitions between states as determined by events. Some of these events come from the outside and are called primary events. The other events are generated by the event processing itself.

It's possible to combine microservices and events without orchestration of any sort, provided that all the events are self-contextual, which means that there's no variability in what process handles the events. Functions can generate secondary events that activate other functions in this model, but that won't work where a real event context is required. Orchestration or step function systems provide a diagram that describes how events move processes through distinct phases and how they provide the data that has to be maintained between process activations, since traditionally functional or microservices programming is stateless.

But where and how will you host your microservices? It's very common to automatically connect event processing and serverless computing. Any cloud-hosted or on premises-hosted application, whether running on bare metal, in containers or in virtual machines (VMs), can access microservices for event processing. In fact, it's very possible this could be the best approach.

Serverless computing is ideal when the scale of event generation includes unpredictable fluctuating time periods of high volume and low volume. This kind of distribution fits a serverless model because you only pay for the service when it's actively processing events. On the other hand, this unpredictable demand makes it expensive to line up traditional cloud instances and keep them ready continuously. Most of the time, there will be either too many, which will generate too high of a cost, or too few, which will reduce performance overall. Where event loads are consistent, other options can be cheaper and offer better performance.

Perhaps, the best option is to write event-driven microservices to run in containers. Containers can be hosted in the cloud -- on container hosting or even simple IaaS services -- and on premises, and containers are efficient in their use of memory. Container microservices are always loaded and available to run, so the usual delay in loading and running serverless apps won't be a problem.

One example uses Spring Cloud and Spring Statemachine to provide event orchestration using software tools that can build microservices in containers and then host them anywhere containers can be deployed. The Saga design pattern can also be used to build event-driven microservices that process each message in a transaction as an event and contextualize them to ensure reliable access and updating.

The reason why running microservices in containers is usually a better choice than VMs is the lower overhead. VM images contain the OS, and containers hold only the applications and middleware. You can generally run two to three times as many microservices in containers on a given server as opposed to VMs. However, containerized microservices are sometimes seen as less secure, so be sure to check with internal compliance specialists for guidance.

The fundamental question for event-driven microservices is the extent to which the events are contextual, or part of a sequence of things that must be considered as a whole. If they are not, then event processing is simply a matter of directing events to the right process point. If they are, which the case is usually, then the context of the events has to be established before, or by, microservices. If you take this approach in analyzing your event-driven apps, you can make the right microservices choice for implementation.