shyshka - Fotolia

Incorta improves direct data mapping for data lakes

Bringing disparate sets of data into data lakes for data analytics hasn't always been easy. Incorta tries to do it with its new 4.6 platform update.

Data analytics vendor Incorta released its 4.6 platform, with new capabilities for data lake storage and management.

Cloud data lakes are an increasingly common way for organizations to store and manage data, especially for data analytics use cases. A data lake is essentially a large data repository that stores data from different sources. More enterprises are using cloud lakes in part because of the cost efficiencies they provide for large volumes of data.

Incorta, based in San Mateo, Calif., is now looking to benefit from the data lake trend with enhanced data lake support within its unified data analytics platform.

Incorta unveiled the new release Feb. 6, but the updated platform became generally available Jan. 28 because Incorta wanted to get customers using it before the public introduction.

While Incorta can be considered a data analytics vendor, 451 Research analyst Matt Aslett said the company is part of a more expansive category.

"At 451 Research we see Incorta as a prime example of what we are calling an enterprise intelligence platform [EIP] thanks to its combination of data ingestion and integration, database management and analytics functionality," Aslett said.

Big name vendors such as Microsoft and Oracle also are moving towards providing an EIP, Aslett noted. However, Incorta has differentiated itself from the mainstream data management and analytics providers with a focus on real-time analytics based on its Direct Data Mapping Engine for performing real-time joins on ingested data, he said.

Incorta data connector screen
Incorta data connector screen

"In terms of the new functionality in Incorta 4.6, the ability to access data in cloud storage services further expands the company's addressable market and positions the Unified Data Analytics Platform for managing and providing access to data across multiple data processing platforms and cloud providers," Aslett said.

Direct Data Mapping accelerates data queries

The founding vision behind the company was to create a unified data experience, in which users could easily connect to data sources and then get answers from the data, said Matthew Halliday, co-founder and vice president of products at Incorta.

We see Incorta as a prime example of what we are calling an enterprise intelligence platform, thanks to its combination of data ingestion/integration, database management and analytics functionality.
Matt AslettAnalyst, 451 Research

A key challenge in connecting data so that it's useful for data analytics is often performance, which Incorta handles in its platform with a capability known as Direct Data Mapping.

Direct Data Mapping replaces several operationally intensive data profiling processes within a database for deciding how to execute a query. By taking a more optimized linear approach, datasets with a large number of tables can still be queried quickly, Halliday explained.

Incorta 4.6 is a cloud data lake release

The main new capability in the Incorta update is enhanced support for cloud data lakes including Microsoft's Azure Data Lake System (ADLS Gen2) and AWS S3-based data lakes.

As an example, Halliday said that if a user wanted to port their Salesforce data to Incorta, they can now bring that data to ADLS Gen2 with a Wizard-based interface that loads the data without the need for custom coding. The Incorta 4.6 platform will now also manage the incremental updates to ensure that data lake data reflects what is current in the original source at Salesforce.

Halliday noted that in earlier releases of Incorta it was possible to bring data into data lakes, but it was a more manual and cumbersome process.

Data Lake use is growing

Cloud data lakes are not a replacement for transactional databases, but some Incorta customers are using them as a form of file-based database, which can be queried. Cloud data lakes are also being used instead of data warehouses, because of the ease with which data can be connected, according to Halliday.

Halliday said that Incorta will continue to focus on increasing speed both for query performance and data ingestion. Another area of future development will be in further improvements to the user experience and data workflow.

Next Steps

Incorta 5 advances direct data mapping technology

Upsolver raises $25M for no-code data lake platform

Dig Deeper on Database management