Big data cloud is changing familiar database tech tune
New cloud database applications that increasingly fall into the "big-data-intensive" category are shaking up the usual build-or-buy soliloquy.
"To build or to buy" has long been the soliloquy of enterprise technology leaders. But as cloud databases and database as a service (DBaaS) offerings continue to mature, administrators are left with more to consider, because the cloud -- and the big data cloud, in particular – is bringing a "to rent" option into the mix.
Storing data in the cloud was by no means the first thought when cloud computing arose. The concept has more than its share of security and compliance issues, and moving masses of on-premises data to the cloud isn't easy. But, for some types of data-intensive applications, rented cloud architecture increasingly seems the way to go.
Databases in the cloud can be hard to isolate from their related apps, which in turn are often buried in vast distributed systems. But market research company MarketsAndMarkets has estimated the cloud database and DBaaS market will reach $1.07 billion this year and grow to $14.05 billion by 2019.
For its part, analyst firm IDC said 75% to 80% of new cloud apps will be "big-data intensive." It isn't hard to accept that projection, as so much of the new data is being generated by e-commerce transactions and other Web applications involving large amounts of data that already happens to sit outside the walls of the enterprise. The growth of mobile applications only accentuates the move.
Let me count the use cases
A look at the use cases uncovered in recent SearchDataManagement reporting shows that data management methods appearing in new cloud architectures have ventured far from the confines of familiar enterprise applications, and, in many cases, are making use of emerging NoSQL and NewSQL database types. For example:
Cloud just makes the opportunities for data explode.Stephen Baier,
vice president of architecture, Fidelity Investments
- An online inventory management operation employed a NewSQL database to handle diverse and constantly changing customer reporting requirements.
- An application performance monitoring service is using NoSQL software on the Amazon Web Services cloud to collect real-time operations data from more than 800 million mobile devices.
- A marketing data services firm found it needed in-memory NoSQL technology to manage one trillion data transactions per month involving online advertising data.
- Another marketing analytics company adopted a NoSQL database service augmented by a SQL-enabled data warehouse in the cloud to query a Web-helping of online and social media data.
One of the big encumbrances to data in the cloud has been physics. "The challenge around big data and the cloud is how much data you have to move, and from where," said Steve Mills, senior vice president and group executive for software at IBM, during a press conference at the vendor's Information on Demand 2013 conference.
While that heavy lifting once limited data use in the cloud, now cloud platforms have broadened to work for virtually any type of application, according to Mills. It's still a matter of ''getting the model straight,'' he said, but his implication is that that modeling effort is well underway.
At the same time, as more data is created in the cloud, it becomes easier to leave it there rather than move it down to on-premises systems.
Cloud weather report
And while it sometimes seems that the big data cloud is the sole province of Web startups, there is movement on the traditional enterprise front, as well. The timing of enterprises' shift to the cloud is still uncertain, but the move is in progress. We heard it straight from a technology leader at financial services giant Fidelity Investments, for example.
Stephen Baier, vice president of architecture at Fidelity, said his company will move at a reasonable pace, but that the fusion of the cloud and big data will eventually be formidable. He spoke as part of a panel last November at a forum on the future of data management held in Boston by Xconomy Inc.
Check out recent Talking Data podcasts
Hear about IDC's take on big data trends
Ponderthe potential of cognitive computing
Learn about the "SQL-ization" of non-SQL databases
"Cloud just makes the opportunities for data explode," Baier said, while noting that there is much work to do. He said the focus now is on private clouds and hybrid public-private variants, but he thinks the reluctance to deploy data-intensive applications on the public cloud will ebb going forward.
"The public cloud certainly has a role in our future," he said. "Mission-critical applications will remain on-premises for some while, but I don't think that holds if you go out five or 10 years."
A lot of the work in moving data to the cloud has to do with re-thinking basic premises that data architects have grown up with. In Baier's words, enterprises are "addicted to relational databases." For cloud database services to grow to their full potential, that needs to change.
While the typical Web startup proceeds somewhat frenetically, the pace of progress for an enterprise's cloud data will be more measured. It will also be moderated by the ability of cloud databases and services providers to meet unique requirements. The original premise of the cloud was based on repeatable mass production, and that too will have to change.
Ultimately, as in the real estate market down here on terra firma, whether to build, buy or rent databases will be calculated by looking at features and price points, and then estimating potential value and deciding what best fits an organization's data management lifestyle.
Jack Vaughan is SearchDataManagement's news and site editor. Email him at [email protected], and follow us on Twitter: @sDataManagement.
