Data companies vie to fill spots in AWS cloud data lineup
Third-party vendors that offer data platforms to AWS users tout hedges against cloud lock-in. But they must both compete and collaborate with the cloud leader.
While the AWS cloud data management lineup continues to expand and gain attention, the Amazon cloud's dominant position in the cloud computing market makes it a prime location for database and data management products from other vendors as well.
At its re:Invent 2018 conference last month, AWS released updates to Redshift, Aurora and other data platforms, at the same time as it entered the time-series database market with Timestream and added a transaction ledger database called QLDB. The product blast overshadowed new cloud services from others.
Nonetheless, many third-party data players were on hand at the event in Las Vegas, taking part in a showcase of co-opetition that has them both cooperating and competing with AWS.
As AWS is the established leader in online commerce and cloud platforms, many other software vendors are effectively consigned to working with AWS if they want to grow their business in the cloud. In the third quarter of 2018, according to Synergy Research Group, the AWS cloud infrastructure services business held a 34% share of a $17 billion market. That is a bigger share than its next four competitors combined.
So, vendors of databases, data warehouses and other data tooling were at re:Invent looking to complement, supplement or supplant software in the AWS cloud data lineup. A common meme at the show held that AWS was widening the market for everyone.
Driving PostgreSQL use
The AWS cloud has played a role in driving interest in open source databases, said Ken Rugg, chief product and strategy officer at EnterpriseDB, which sells software and services based on the PostgreSQL open source database.
"To some extent, PostgreSQL has become more popular in the last couple of years because people are using it on Aurora and RDS," Rugg said. These two AWS database services both sport PostgreSQL compatibility.
At re:Invent, EnterpriseDB launched the EDB Postgres Cloud Database Service, with Oracle-database-compatibility, running on AWS. It is initially available in two AWS computing regions.
Like other data product vendors, a benefit EnterpriseDB holds out for users is support beyond the AWS cloud. That means both on-premises support and coverage on other providers' clouds. "People don't want to get locked in," Rugg said.
Data integration moves to the cloud
Also at the show, enterprise data integration specialist Informatica introduced a cloud package that matches its data cataloging capabilities with Tableau visual analytics and Amazon Redshift running on AWS. For many users, the Redshift cloud data warehouse has been a first step toward enterprise data migration.
Today, people choose the cloud first. Cloud is not an 'if,' it's a 'when.'
Matt Glickmanvice president of customer and product strategy, Snowflake
"AWS is leading the market in terms of moving data and processing to the cloud," said Ronen Schwartz, senior vice president and general manager for data and cloud integration at Informatica.
Still, Schwartz sees gaps in AWS's approach -- ones that Informatica will try to fill.
"We understand our strengths and weaknesses," he said. "We have very different users."
Schwartz noted the abundance of software developers at the big AWS event. In fact, Informatica first rose to prominence in enterprise data warehousing with data transformation tooling that reduced the need for developers to do prodigious program scripting.
"The main [AWS] user is a developer," he said. "Coding and development have always been in competition with our approach."
Rather than target products at developers, he said, Informatica's products target data engineers. "This allows us not just to coexist but also to strengthen each other," he said, referring to AWS.
Massive scale sold here
Massively scaled cloud computing infrastructure is an AWS specialty, and purveyors of data software ranging from Hadoop data lakes to analytics-oriented data warehouses have to be there. Even a generously funded data warehouse startup such as Snowflake Computing is compelled to find a form of coexistence with AWS.
"Today, people choose the cloud first. Cloud is not an 'if,' it's a 'when,'" said Matt Glickman, vice president of customer and product strategy at Snowflake.
Customers may be ready to go to the cloud, but they also are aware of the potential for lock-in when dealing with a big cloud provider. Like EnterpriseDB's Rugg and other AWS partners, Glickman points to the growing use of hybrid clouds and future use of multi-cloud architectures as reasons to consider non-AWS cloud data alternatives.
Meanwhile, Glickman aims to differentiate the Snowflake cloud data warehouse from Amazon's Redshift platform. Concurrency is the main distinguishing factor, he said, referring to the concurrent users that contend for query time as they ask business questions of a shared data warehouse.
Cloud is the place
Wayne Eckerson
The cloud is increasingly the destination for the data lake and the data warehouse, and Snowflake has found a place in the mix, despite wide use of Amazon Redshift, said Wayne Eckerson, founder and principal consultant at Eckerson Group.
At its heart, Snowflake has an architecture that separates storage and data processing, Eckerson said, and this helps support high concurrency for user queries.
"Snowflake can work with S3 [cloud object] data, and give you a data lake plus a data warehouse. It is very flexible," he added.
Eckerson noted as well that Amazon is rearchitecting Redshift, and signs of that were to be found at the AWS conference.
AWS cloud data shifts
At the show, AWS disclosed Amazon Redshift Concurrency Scaling, as part of an effort to independently scale storage and computation, which looks to cover some of the same ground that Snowflake has claimed with its software.
Tools like Amazon Redshift Concurrency Scaling speak to the nature of co-opetition today. AWS partners look to keep innovating, as they know AWS is on the lookout for features that customers want to add to data portfolios.
But smaller players like Snowflake aren't standing still. Just ahead of re:Invent, the startup released a version of its software that supports automatic clustering and materialized views. This reduces administrative overhead and helps fine-tune data warehouse user queries.
As in other technology relationships in the past, AWS's stance toward third-party data software makers has the larger vendor offering general-purpose platforms, and then responding to new capabilities that its smaller rivals provide. This all makes for a continuous stream of interesting developments for customers to follow.
Wayne Eckerson
