Use event processing to achieve microservices state management
What good are stateless microservices for stateful processing? Bring complex event processing into the mix to manage microservices state through scaling and failures. Here's how.
Stateless microservices do not store internal information from one execution to another. As a result, they do not track the user interaction dialog or maintain process state information. This can be a big problem.
When state is relevant, this information has to persist. However, it can be lost when a microservice goes through flexible changes, such as when it scales up, gets replaced or scales down.
It's possible to limit your use of stateless microservices to cases where state is not relevant. You could also code state control directly into the microservice, but this approach adds the risk that the services become less agile and portable.
Instead, consider microservices state management with events. An Event provides state information for those microservices through a comprehensive event-processing model. Let's take a look at the event-processing strategies that can help maintain state for stateless microservices, including the back-end, front-end and orchestration models.
Combine event processing and state
The traditional event processing model queues events as received, processes them in order and triggers the appropriate actions within the application. However, the relationship between events and state management is a little more complex. With microservices adoption on the rise, organizations must delve into state management thoroughly.
If an event is simple, meaning it doesn't rely on context information to perform its duties, the application can process it immediately. But if the relational context of the event matters, a method known as complex event processing (CEP) comes into the picture. Even if a series of events derive from the same source, the application must interpret or handle each given event based on information pertaining to state.
Distributed cloud-hosted applications prompt a change in event processing, in which events trigger transient processes and send them throughout the cloud according to where the event originated and where action is needed. In other words, this processing model interprets transactions as a sequence of connected events rather than as a fixed multistep workflow.
Three specific issues emerge pertaining to microservices state management. First, when complex event transactions are treated as a series of events, the application architect must figure out how to introduce state without compromising the inherent statelessness of microservices. Second, they must find a way to communicate the dialog state or the interpretive context for an event to the stateless microservice. Finally, the application must be designed so that any data associated with the event or process flow gets stored and provided to the microservices, matched to the correct event context.
Front- and back-end control
To solve these problems, create some form of context ID, which can act as a unique transaction identifier or event flow ID you assign when an event launches a contextual process. This identification can come in many forms, but it doesn't need to be complex. For instance, a simple timestamp for when an event occurred can serve as a context ID. When the application attempts to track the steps that occurred in a user dialog, it can create a unique ID by referencing a timestamp.
Unfortunately, it's often confusing whether a process close to the event source actually maintains the state itself, or whether that state is somehow provided from outside the service. For this reason, it's essential to create unique transaction IDs for all state-dependent operations. You can use that ID to retrieve specific state information from a database or to drive transaction-specific orchestration. This ID also helps carry data from one phase of a transaction, or flow, to another. This setup is essentially a back-end approach to microservices state management.
Front-end state management is another option. Control from the front end means that a user or edge process maintains the state data. This state information passes along through an event flow, and each successive process accumulates more information from the previous ones. Since this state information queues along with the events, you won't lose state data during a failure as long as the event stays intact. Also, when processes get scaled with more instances of the microservices involved, the event flow can provide the state data those processes need.
Source routing and orchestration
Front-end state control approach requires a form of source routing. This means that any event-related process must provide information about the next process in line to maintain the necessary event flow. Back-end state control, where state and other data is retrieved by a process as needed, also requires that the source event provides some information about the next upcoming event.
To ensure that event orders stay aligned, adopt event-processing orchestration as part of the overall microservices state management strategy. This orchestration process compiles the stateless elements to create a stateful service. This resulting stateful service defines the service workflow model and takes authority over when each service is invoked. By providing a predictable transaction sequence for messaging between services, the model can guide event execution and effectively maintain state. While few cloud providers offer this type of orchestration as part of their portfolio -- like AWS Step Functions, for instance -- open source options exist like Netflix Conductor and Zeebe.
