Qlik unveils pair of new integrations with Databricks

New integrations

Databricks Lakehouse (Delta) Endpoint and the integration of Qlik Cloud with Databricks Partner Connect -- both launched on Sept. 26 -- represent the latest collaboration between the vendors.

Databricks Lakehouse (Delta) Endpoint is a tool in Qlik Data Integration that takes advantage of new SQL-based interfaces from Databricks to enable joint users to ingest data with Qlik Data Integration and transport it into the Delta Lake on Databricks in real time. A previous integration between Qlik Data Integration and Databricks did not feature the SQL-based interfaces.

Meanwhile, Qlik and Databricks also have an integration between Qlik's analytics tools and Databricks' lakehouse that enables joint customers to access and analyze their data already stored in Databricks.

The integration between Databricks Lakehouse and Qlik Data Integration is important in ensuring that Qlik customers can leverage their existing investments and skills to integrate data into the Databricks Lakehouse.

It's that ability to use Qlik both before and after data is stored in Databricks that makes Qlik's integrations -- including the evolution of Databricks Lakehouse (Delta) Endpoint -- significant, according to Matt Aslett, analyst at Ventana Research.

"The integration between Databricks Lakehouse and Qlik Data Integration is important in ensuring that Qlik customers can leverage their existing investments and skills to integrate data into the Databricks Lakehouse," he said.

Aslett added that demand for data lakes is growing, and with that growth comes the need for analytics and data integration vendors to ensure that their tools work with those of data lake and data lakehouse vendors.

"Data lake environments predominantly coexist with existing data processing and analytic investments, so it is imperative that data lake environments can be used with both customers' existing data integration and their analytics products and services," he said.

The result of the new integration is a more efficient data ingestion process than was available with the previous integration, according to Itamar Ankorion, senior vice president of technology alliances and managing director of enterprise data integration at Qlik.

Beyond enabling Qlik customers to use their existing tools with Databricks throughout the analytics process, Databricks Lakehouse (Delta) Endpoint is important for Qlik users because it more smoothly enables integration with a lakehouse that is optimized for data science, according to Donald Farmer, founder and principal of TreeHive Strategy.

In addition, Qlik's array of connectors enables the integration of complex data in Databricks so that users can perform analysis on more complete data sets, he continued.

"The Qlik architecture -- especially its wide range of replication and changed-data-capture connectors -- enables the Databricks lakehouse to incorporate heterogeneous data sources, including legacy sources, which can be very difficult to integrate in any other way," Farmer said.

Meanwhile, the integration between Qlik Cloud and Databricks Partner Connect is aimed at enabling Databricks customers to try out Qlik and experience how the platforms perform together.

The development of both new integrations resulted from a combination of Qlik's desire to take advantage of the latest technology from Databricks -- the SQL-based interfaces, in the case of Databricks Lakehouse (Delta) Endpoint -- and customer feedback, according to Ankorion.

"As Databricks' product evolved with new capabilities, we made the investment to align with them to provide more value to joint customers," he said. "In addition, customers always prefer and ask about cost-performance optimizations."

Comparing cloud connectivity

As more customers migrate their data to the cloud, many analytics vendors have made efforts to enable those customers to use the cloud of their choice by developing connectors and integrations with the various data clouds.

For example, SAS, though its platform is compatible with most major clouds, has developed a close partnership with Microsoft Azure and continues to add functionality in concert with Microsoft. Even smaller vendors like Toucan Toco are aligning themselves with the various cloud data platforms.

But with data integration, Qlik is among the furthest along in connecting with the likes of AWS, Azure, GCP, Databricks, Snowflake and others, according to Farmer.

The vendor's platform doesn't offer scenario planning tools. And it could improve its extract, transform and load capabilities, analysts have said. But for connecting data integration tools with cloud data warehouses and data lakes, Qlik has been forcefully moving ahead as a result of its 2019 acquisition of Attunity.

Attunity was the first third-party data integration vendor to integrate with Amazon RedShift, dating back to 2013, just after the launch of the tech giant's cloud data warehouse, Farmer noted.

"Qlik therefore has more experience of cloud data integration than any other vendor in the market," he said. "And it shows in the robustness, performance and thoroughness of their offering."

Looking forward, Farmer added that he'd like to see more connections between Qlik's data management and administration tools and the cloud service providers.

"Many cloud data platforms are weak in administration and management, and Qlik can fill that gap," he said.

Ankorion, meanwhile, noted that Qlik is indeed working with the various cloud data platform providers, including Databricks, to add more integrations, though he did not name the specific Qlik capabilities involved in the next wave of those integrations.