How to create a stateful application with service brokers, DTP
Complex state control issues come with the territory of stateful application design. Learn useful concepts that manipulate state and enable a stateful app to work as envisioned.
Transactions represent steps in the sometimes complex set of commercial exchanges that move goods and services, pay invoices and taxes and otherwise make business happen. Basically, transactions represent exchanges that adjust balances and inventories appropriately. Transactions are inherently stateful, because each change to a balance or inventory affects the transactions that follow.
Stateful applications rely on either inherently stateful services or microservices, or on a separate mechanism that maintains state. The latter is a method to manipulate stateless services and microservices to work in a stateful way through external state control.
Dialog state and process state
Before investigating internal and external state control for application services, make sure to understand how stateful transactions happen. There are two states in any transactional system: dialog and process.
The dialog state represents the natural multi-step review sequence of transactions. This type of state is reflected in the number of steps involved in the user's interaction.
The process state determines whether new copies of an instance of a service -- such as for scaling or failover -- impact data integrity or break the interaction between a user and the application. Developers of stateful applications must mitigate process state limitations on scaling and replacement, while managing the dialog state.
One fact about the stateful service approach that can confuse developers is that stateful application diagrams often use the same service or microservice terminology as stateless examples do. A natively stateful service, in a process-state sense, is one that contains the internal variables or database storage associated with it, and it alters the associated data upon each transaction. You cannot simply spin up a new instance of a stateful service, or replace a failed one, without considering how to recover or synchronize that stored data.
The storefront design pattern
Stateful services usually run behind a pattern that is similar to a storefront design, which offers users a view of the things they can do and reacts in accordance with their choice. Each choice represents a stateful service. This development model encourages programmers to separate applications into feature-based services. They can use a database service broker, either integrated with or behind the storefront, to manage the health of each service and redeploy them as needed. A service broker can also provide access control, based on sign-on credentials for example.
A microservices design pattern combines an API gateway for mobile application users with a storefront pattern for browser-based users. It uses this combination because mobile apps provide the GUI within the mobile device and then connect directly with the features behind the storefront. In this scenario, the storefront should access features through the same broker regardless of user type, to ensure that all services are deployed, scaled and replaced in the same way. This also presents the same set of features to both mobile device- and browser-based users.
Services and queues
It's often necessary to associate queues with services. Feature services can have fairly long execution times, which means they'll be busy for extended periods of time. You can theoretically introduce the queues at the service-broker level or on a per-service basis in stateful application workflows, but the queuing technology must be resilient to prevent losses in the case that a service instance fails. Introducing a queue at the broker level is effective because it eliminates the problem of managing per-service queues if you scale instances.
Don't build per-service support for service registration, replacement or scaling in a stateful application. It's possible to convert the basic stateful service deployment model to one that includes multiple service instances to manage variable workloads or replace failed components. The service broker can act as a stateful load balancer, which recognizes multi-step transactions and ensures that all the steps of that given transaction direct to the same service instance. Another alternative is to create feature services for each of the dialog steps, which circumvents the issue of a service handling multi-step transactions. For example, a separate view-and-update service can support inquiry via the view capability and help execute updates in sequence.
DTP and serialization
With a scalable service, every business transaction must trigger a database update that reflects the changes, such as altered account balance or item inventory. That's an inherently stateful change, and you can't have multiple instances of a service making overlapping changes to databases. Distributed transaction processing (DTP) and serialization both address this issue.
DTP locks a database in between the retrieval of a record that's set to update and the actual update process. One common approach to DTP is a two-phase commit (2PC), which performs an automated lock, update and release process. Many relational databases inherently support DTP as a part of their atomicity, consistency, isolation and durability property, referred to by the acronym ACID.
Serialization uses a message queue as an on-ramp to a stateful application service, performing transactional database operations. Primary functions queue work items for database action at the appropriate point, and the queue holds the items so the processes commit in the proper order. Developers must explicitly design applications to use this mechanism properly, or risk errors.
Contextual multi-step dialogs between users and applications are fundamental to business, so it's essential to use some mechanism to handle dialogs statefully. Stateful services work in a way that's familiar to application developers, but that is potentially not optimal for cloud-based hosting. Consider options that ensure microservices scale and redeploy on cloud resources for flexible stateful applications.
