Sergey Nivens - Fotolia
MongoDB Atlas Online Archive brings data tiering to DBaaS
MongoDB's online archive service gives organizations the ability to automatically archive data to lower-cost storage, while still enabling the data to be queried.
MongoDB today released into general availability its Atlas Online Archive service, providing users with a service that enables data tiering.
With data tiering a user can archive older, less active data on lower-cost archival cloud storage such as Amazon S3, while active data that is more often accessed and queried remains in the primary database.
MongoDB Atlas is the New York-based vendor's cloud database as a service platform (DBaaS). MongoDB previewed the Atlas Online Archive in June, alongside the MongoDB 4.4 update. A key benefit of the MongoDB Atlas Online Archive is that it's easier for organizations to now use MongoDB to query data inside the database, as well as data that is outside of it.
Among the organizations that have been using the MongoDB Atlas Online Archive in preview is installment payment provider Splitit, which is also headquartered in New York.
Splitit CTO Ran Landau said he has a strong focus on observability and wants to know everything that is happening with his company's systems. Splitit collects a lot of data that helps to inform operations as well as business intelligence.
Landau said that in the past, Splitit has used various approaches to enable data archiving. None of them were particularly easy or efficient.
_half_column_mobile.jpg)
"I couldn't use the data that I wanted to use, and then I moved to Atlas Online Archive and I finally could use all of the data," Landau said.
He said that for many operational use cases, Splitit only needs fresh data, which is 30 days old or less. Splitit, however, uses the older data for business intelligence to help inform historical and longer-term trends.
As a managed service, MongoDB Atlas autoscales based on database load and the Atlas Online Archive automatically moves data once it's older than 30 days, Landau said.
How MongoDB Atlas Online Archive data tiering works
Sahir Azam, chief product officer at MongoDB, said the major value of Atlas Online Archive is data tiering. It enables users to cost-effectively store larger volumes of data and benefit from insights that data might provide.
"We've seen a lot of customers tell us there's data they wouldn't traditionally be able to keep in a database online, where it can be easily queried because it's just a lower-value older data set that, frankly, they would often delete, or dump into some sort of archival storage where it was inaccessible," Azam said.
MongoDB users aren't just archiving data based on date. Azam said that over the preview period, users asked for the flexibility to create customized policies for archiving.
So now in the GA release of MongoDB Atlas Online Archive, users can create a policy that can archive a subset of data that could be based on any number of different criteria. For example, a user could choose to archive all data that comes from a certain class of device, or all data from IoT environments.
Performance improvements in future MongoDB Atlas Online Archive updates
MongoDB has also seen early users of Atlas Online Archive using the service as part of complex data engineering workflows.
Azam said that among those use cases are organizations that want to preprocess data in some way before it is archived. He added that MongoDB is currently working on enabling more automation and flexibility for data engineering.
Performance improvements is another area that MongoDB is working on for the Atlas Online Archive service. The vendor's goal is to enable faster query speeds for data sets that are in the archive.
"If you think about the database data as hot and our current online archive is sort of colder data, we think we can create something in the middle," Azam said.
