Can you really use a shared database for microservices?
Since its emergence as a mainstream development style, experts have debated the wisdom of using a shared database for microservices versus providing a database per service.
For years, experts have stressed that the success of a microservices architecture rests upon bounded contexts, which draw clear boundaries between underlying functional logic and application state information. Part of fostering this proper separation of concerns involves moving from a centralized database structure to one where each service is assigned its own designated data store.
While this approach promotes service independence, loose coupling and failure isolation within the architecture, it is important to consider the tradeoffs: namely, the level of management complexity and resource demands this type of structure may impose.
Let's review the benefits -- and pitfalls -- of using a database per service versus a shared database for microservices. Then, we'll examine whether the shared database approach presents value despite its potential drawbacks, and where it might even be the ideal design choice.
The database-per-service approach
The database-per-service approach is one that aims to provide numerous instances of expressly targeted functionality. Database choices can be polyglot, and service can dependably operate in accordance with a particular data schema, scale against relative throughput rates and adhere to the database's read/write characteristics. Each service can scale independently, and any failures or slowness caused by neighboring services aren't likely to spread across large portions of the system.
However, providing one dedicated database for each service presents several challenges, especially in large-scale enterprise systems with complex communication and integration requirements. For one, it will take a lot of time and effort to perform large data joins across network boundaries. The proliferation of all these databases also increases the risk of extensive, resource-draining data redundancy, especially if a team fails to adequately define bounded contexts. Tying multitudes of databases to a large stack of independent services will also make it difficult to consistently maintain synchronized application states.
The shared database approach
The most direct way to sidestep the pitfalls of a database-per-service approach is to instead provide a single database from which multiple services can pull necessary resources. For instance, using a shared database makes the process of combining disjoint set data structures much more straightforward. As long as all required tables live within a single database, distributed transactions can safely execute through the use of atomic guarantees and database primitives.
Of course, shared databases come with their own share of pitfalls. As shared database schemas evolve over time, all the services that depend on that database will need to update in accordance with the schema change. Another inconvenience is that not all services sharing a single database will necessarily use that database in the same way, and configuring the database for one service may prevent another from doing its job. For instance, in large-scale software systems, a single shared database may not be able to provide the levels of data volume and throughput demanded by every service, which may force IT to split the database unexpectedly.
A shared database will also suffer when dealing with high write volumes, and compaction of segment tables may cause significant drops in performance. This is a situation that resembles the classic noisy neighbor problem generally associated with all types of shared application resources.
When to use shared databases with microservices
While the downsides may seem to nullify its practicality, there are still situations where the shared database approach is an excellent fit for microservices. For one, it is an effective pattern for incremental migration away from complex, monolithic systems. In terms of implementation, most modern object-relational mapping and web application frameworks readily allow multiple services to use databases concurrently without interrupting operations.
Here are some of those scenarios for maintaining shared databases between microservices:
Breaking up a monolith
When migrating a monolith to microservices, it's ideal to start by breaking collections of underlying business logic into independent microservices within a shared database. As the logic is split into independent units, the tables within the shared database can be logically partitioned to clarify the separation of logic and data.
Once those tables are properly partitioned, it's possible to then introduce the database-per-service model. Because a transition like this can last months -- or even years -- to complete, it makes sense to allow a limited number of services to share databases even as other services are assigned their own individual database.
Strong consistency requirements
The shared database approach is also helpful when an architecture's primary requirements revolve around data consistency. When an architecture demands strong data consistency, despite having multiple services, it may help to create a shared database that provides locking and synchronization at the database level. Organizations can enforce this approach by either including it within the business requirements handed to development teams or by making it a requirement in architecture design guidelines.
Low volume and high interdependency
Complications with shared databases are not nearly as problematic when dealing with low volumes of data. So long as that shared database maintains logical table partitioning between logic and state information, it offers a much nicer alternative to the technical complexity associated with managing multiple databases. Even if the application or number of services will grow over time, the shared database can still act as a good starting point until the system reaches a critical mass.
This is especially true for low-volume systems that also demand a certain degree of coupling between services. It makes sense to continue using a single shared database in this architecture because you'll face much less refactoring and application redesign work.
Techniques to support shared databases
As mentioned earlier, shared databases introduce tight coupling between services, so they should be used with caution. It's also important to examine whether a perceived need to explicitly share a database between services simply stems from an inability to clearly define bounded contexts.
However, assuming those bounded contexts are properly defined, there are a few techniques that can help teams mitigate the unwanted side effects of using shared databases for microservices.
Logical partitioning of tables
Even if microservices share a database, it is possible to configure a single database so that tables are separated by clearly defined, logical boundaries and owned by specific services. There are simple ways to enforce this separation, such as assigning database-specific roles and permissions to individual services. This can help prevent accidental access or unwanted coupling between services. You can also create certain communal permissions that are used when multiple services need to share a single set of data.
Single writer, multiple readers
It's also possible to strategically use hot tables (a term used to describe tables shared by multiple services) to support a shared database model. One way to approach this is to allow one service in that table ownership over all data writes and update processes. Meanwhile, multiple services can continue to read data without any effect on the underlying writes and updates. In some cases, it may be possible to do this through change data capture events.
