Starburst acquires Varada to accelerate data lake queries
After a year of partnering, the data lake query vendor decided to acquire fellow Trino SQL query engine supporter Varada to help boost query performance.
Data lake query engine platform provider Starburst acquired query acceleration vendor Varada. Financial terms of the deal, made public on Thursday, were not disclosed.
Varada had raised $12 million and introduced its data platform in December 2020, with an emphasis on query acceleration.
The two vendors had unveiled a partnership in March 2021; it went so well that Starburst's management wanted to acquire Varada.
With the acquisition, Starburst can now directly integrate Varada's technology with its own Enterprise on-premises and Galaxy cloud services to accelerate data queries. Both Starburst and Varada are active participants in the development of the open source Trino query engine technology.
The market for data lake query and management technology is highly competitive with multiple vendors in the space, including Presto-based Ahana, Databricks with its Spark SQL data lakehouse technologies and Dremio and its Sonar query engine. A key challenge Starburst and its competitors face is the need to provide data governance in data lakes.
Opportunities and challenges in the data lake query engine market
The acquisition is a good move for Starburst, said Matt Aslett, an analyst at Ventana Research.
The combination of workload monitoring, autonomous indexing and query orchestration that Varada brings to the acquisition should enhance Starburst's ability to deliver accelerated analytics on data lake environments distributed across cloud and on-premises infrastructure.
The combination is highly complementary. It adds further data lake analytics acceleration capabilities to Starburst's platform through Varada's caching and indexing functionality, as well as its workload observability capabilities.
Matt AslettAnalyst, Ventana Research
"Varada was already working with Trino and was a Starburst partner, so the combination is highly complementary," Aslett said. "It adds further data lake analytics acceleration capabilities to Starburst's platform through Varada's caching and indexing functionality, as well as its workload observability capabilities."
Data lake query technology options for organizations are expanding, but data lake speed performance compared with moving data into and out of databases or data warehouses remains a challenge, said Doug Henschen, an analyst at Constellation Research.
"This deal helps Starburst step up performance with Varada's caching and indexing technology," Henschen said. "The deal closes a gap in Starburst's technology and team, enhances the solution's appeal to would-be customers, and steps up competition with direct Starburst competitors, including Dremio and Ahana."
Why Starburst bought Varada to accelerate data lake queries
One of the key factors in Starburst's move to acquire Varada was Starburst CEO and co-founder Justin Borgman's expectation that the vendors' technologies could be integrated relatively quickly.
Because the vendors have been partners for more than a year and use the Trino SQL query engine, Borgman said he expects it will take only 90 days for the Varada technology to be integrated into Starburst.
"That's pretty lightning fast as far as acquisitions and integrations go," he said.
In terms of what Varada brings to Starburst, Borgman said Varada's value is that it can make Starburst user queries up to seven times faster, which can help reduce costs.
Organizations that run Starburst Enterprise, which organizations deploy on their own infrastructure, will get better performance with less infrastructure, as less hardware will be needed to achieve performance goals, Borgman said.
At the core of Varada's platform -- and the main reason Borgman said he was interested in the company -- is the smart indexing caching technology that enables the query acceleration.
Varada's technology caches queries based on workloads. The Varada smart index watches query patterns and automatically decides what to cache to enable performance improvements.
Data lake analytics and more speed
As to why faster queries are important, Borgman said it has to do with larger trends in the data lake arena.
The key trend is the shift toward data lake analytics, an area some vendors refer to as a data lakehouse, Borgman noted.
The basic idea is to enable the use of data lakes for analytics without needing to first load all the data into a data warehouse.
Starburst's acquisition of Varada and its technology is a step toward reducing or eliminating any performance differences between a traditional data warehouse and a data lake engine, Borgman said.
That data lake performance boost is particularly useful now as many observers anticipate a recession, worry about the effect of rising interest rates and customers become a bit more cost-conscious, Borgman said.
"We think this is a really compelling way for them to reduce their analytics costs by moving workloads to a data lake, and still getting great performance out of it," he said.
