polyglot persistence

What is polyglot persistence?

Polyglot persistence is a conceptual term that refers to the use of different data storage approaches and technologies to support the unique storage requirements of various data types that live within enterprise applications. Polyglot persistence essentially revolves around the idea that an application can benefit from using more than one core database or storage technology.

Polyglot persistence suggests that an organization's database engineer or architect should place a priority on figuring out how they'll need to manipulate application data and identifying the database technology that will best fit those needs. This approach is advocated as a way to address storage performance issues, simplify data operations and eliminate potential fragmentation upfront.

Core tenets of polyglot persistence

Polyglot persistence draws from many of the same ideas encapsulated in polyglot programming, which refers to the practice of writing applications using a mix of languages. The idea is that this allows developers to take full advantage of each various languages' suitability for solving different types of application development and management challenges.

Polyglot refers to the ability to communicate fluently in several languages. In the context of application development, it is used to describe an application's ability to operate and communicate through multiple programming languages. In data storage, the term persistence refers to survived data after the application process it was designed to support is terminated. This data is then stored in a non-volatile storage location.

However, not all organizations are in a place where this approach will benefit their operations, and an enterprise should ask several questions before it makes the decision to implement polyglot persistence. Some of these specific questions include the following:

• How will developers and database managers be trained to work with the new system?
• Is there an expert available who can help get the system up and running?
• Is there existing in-house expertise that can help mentor the rest of the staff?
• Who will provide support and repairs when problems occur?

Once satisfactory answers have been found for each of these questions, polyglot persistence can be implemented to help an enterprise begin to make the move away from singular relational databases and toward a mixture of data sources.

How to use polyglot persistence

Database architects and engineers must design and calibrate their polyglot persistence approach for the specific data architecture that exists within their enterprise. Luckily, polyglot persistence techniques readily adapts to the use of SQL, NoSQL or hybrid database systems.

Various factors should be taken into consideration when deciding to move to a polyglot persistence storage system. For one, it will likely require the addition of a new database -- along with other supporting technologies -- in order to sustain the new implementation. It's also worth determining whether it will be necessary to support ACID versus BASE compliance .

The most important factor to understand is the data flow within the organization. An easy way to do this is to establish an owner for each piece of data in the system. Introducing this detail at the overall system architecture level will allow developers to see which owner is able to modify their corresponding piece of data as well as how this data will be distributed in the system, thus making the work on each of the different pieces of data more feasible.

There are several standalone tooling options available to automate polyglot persistence, such as CloudBoost.io, though large platform providers like Red Hat and Azure also provide support for this approach.

Data storage types

Polyglot persistence makes use of a wide array of data storage types. Here are some examples of the various components and hardware involved:

• External hard drives
• Network-attached storage
• Cloud-based storage
• Solid-state drives
• Document, graph and search databases
• Row-store databases
• Data caches

Polyglot persistence strengths and weaknesses

The polyglot persistence approach carries its fair share of both strengths and weaknesses. Some strengths of polyglot persistence include the following:

• Simplified data management operations
• Reduced amounts of data fragmentation
• Improved response time to data requests
• Increased scalability for applications and systems
• More flexibility in deciding where data is stored

On the other hand, some of polyglot persistence's major weaknesses include the following:

• The need to remodel data systems based on the specific data architecture
• Database engineers and architects become more responsible for data storage
• Database interactions can become more complicated
• New data stores may add a need for more training
• Complex database integration may incur higher operational expenses

Example of polyglot persistence

E-commerce can provide us with a good example of where an organization might apply polyglot persistence. The average web-based retail site often uses many types of data for a shopping cart component: transactional data, session information, inventory counts, order histories and customer profiles.

In the past, an organization might use a single database engine to handle all these different types of data. However, doing so requires extensive data conversion to format the different data types within a single, relational database. Polyglot persistence, however, suggests that the shopping cart (and related e-commerce) data be divided into databases that are best suited for each data type, thus taking the pressure away from one overused data location.