IBM advances its Cloudant cloud database as DBaaS grows

IBM has been in the database-as-a-service market since at least 2014 when it acquired NoSQL database provider Cloudant.

Cloudant's DBaaS is based on the open source Apache CouchDB project, which was one of the first NoSQL efforts. Over the last six years, the market for NoSQL databases has shifted as organizations have required more scalability and features that are often associated with SQL databases, including data consistency guarantees.

To that end, IBM on Tuesday released the new Transaction Engine architecture for Cloudant, bringing improved scalability, native database encryption and enhanced data consistency. It is generally available now.

In this Q&A, Adam Kocoloski, IBM vice president and CTO of IBM cloud data services, and one of the co-founders of Cloudant, discusses the evolution of DBaaS, NoSQL and where cloud databases are headed.

How did you get started at Cloudant and why build a DBaaS in 2014?

Adam Kocoloski

Adam Kocoloski: A couple of folks in the research group that I was part of at MIT thought it would be cool to start a database company together because we felt we had some expertise in managing non-trivial type of global data sets. We pitched startup accelerator Y Combinator in 2008. They thought it was a good idea, and that was the beginning of Cloudant, which was one of the first database-as-a-service platforms. Back then people thought it was a crazy idea that organizations would upload their data to the cloud.

Now database-as-a-service is a very common trend. When we were building Cloudant, I found a ton of value in the work of building and running a cloud offering. When we were going through the acquisition process and talking to IBM, it was clear IBM shared the ambition of building a cloud service provider. IBM's commitment to being a public cloud service provider is now as strong as ever. I count myself fortunate to be one of the small group of people who chart the sort of technical strategy for us to achieve our goals with IBM Cloud.

Beyond Cloudant, what's the IBM Cloud database strategy?

Kocoloski: The IBM cloud database portfolio is a standardized way that we offer support for a whole bunch of data stores and we mostly look at open source. So we've got PostgreSQL, Elasticsearch, MongoDB, Redis and a handful of others as well.

That portfolio is focused on making sure those database engines are accessible in a way that is integrated with the underlying IBM Cloud platform. We want it [IBM Cloud] to be the easiest way for developers that have already chosen a data store to power an application, to be able to provision and build that on IBM Cloud. That's something that involves, in many cases now, deeper partnerships with a lot of the vendors behind those systems.

How does the new Transaction Engine in Cloudant change things for users?

Cloudant was originally designed to deliver cloud scalability, but it did so by sacrificing some isolation aspects of a traditional relational database system. So we set the goal for ourselves to try to figure out if there was a way that we could recover isolation semantics without sacrificing the developer-friendly structure of Cloudant.

Kocoloski: We're giving people an opportunity to build applications that can scale further without having to worry about mitigating the edge cases associated with eventually consistent NoSQL databases. Cloudant was originally designed to deliver cloud scalability, but it did so by sacrificing some isolation aspects of a traditional relational database system. So we set the goal for ourselves to try to figure out if there was a way that we could recover isolation semantics without sacrificing the developer-friendly structure of Cloudant.

Over the past year, we have been working on developing a transactionally consistent, scalable data store. So a lot of that architectural plumbing is all well and good, but the benefit to users is they can now build applications that have much more demanding query requirements. Because we're able to organize the data to satisfy queries efficiently under the hood, we can lift the caps that we would previously put in place on queries.

What's your view of the evolution of NoSQL at this point as vendors increasingly put SQL-type capabilities into NoSQL databases?

Kocoloski: There has been a lot of ink spent on debating the evolution of the term NoSQL. Having scale without sacrificing your consistency is a trend that has shifted in NoSQL. Ten years ago NoSQL developers said it was too hard to do distributed isolation, so the requirement was relaxed to help enable scalability. I think you're seeing a lot of systems these days figure out how to preserve the kinds of isolation semantics that are super important for developers to build trust in critical applications, while preserving the reliability, availability and scalability of these distributed data stores.

Have you seen any change in the adoption of DBaaS in recent months with the COVID-19 pandemic?

Kocoloski: It hasn't been a major inflection point. I think that folks who were already in the middle of a cloud transformation program are in some cases choosing to accelerate those programs in order to realize the business benefits of that transformation program sooner rather than later.

Frankly, we continue to see healthy month-to-month growth and new use cases arise.

What do you see as the future of the DBaaS market?

Kocoloski: I think the nature of being a cloud service provider in 2020 is one where there is a need for a pretty broad portfolio of capabilities.

We take a look at and survey the ecosystem and sort of pick and choose the things that are really meaningful to our enterprise customer base and focus intensively on those aspects.

On the analytics side, we're seeing a lot of people who are embarking on programs to put more of their warehousing data, more of their Hadoop cluster data into the cloud.

On the application development side, you see so many vendors coming with their own cloud DBaaS offerings. I think we're still early in the migration of mission-critical, OLTP [online transaction processing] applications and it's one place where I think the market as a whole is figuring out what's going to be the thing that gets organizations to move. We are starting to have those conversations with customers about migrating those kinds of core transactional systems, but that's not something that happens overnight.

Editor's note: This interview has been edited for clarity and conciseness.