Ascend aims to ease data ingestion with low-code approach
Data platform vendor Ascend has announced a new low-code approach to building out data pipelines on cloud data lakes to accelerate analysis and insights.
The challenge of making it easier to connect different data sources is one that Ascend.io is looking to solve with its flex-code data connectors.
With the new connectors, the company aims to simplify the way different data sources can be connected in its Ascend Unified Data Engineering Platform, which enables users to create data pipelines for analysis. The flex-code data connectors are a low-code approach to connecting data, and they don't require custom coding to enable data ingestion. The new flex-code data connectors were announced as a preview last week, with general availability set for 2021.
Mike Leone, a senior analyst at Enterprise Strategy Group, said he views the Ascend flex-code data connectors as an important development for data ingestion. It's often all or nothing with low-code and no-code approaches to building data pipelines, he said -- either people embrace it and make use of drag-and-drop functionality to quickly build out a pipeline, or they end up writing hundreds of lines of code for every connection and use case. There really isn't a middle ground between the two extremes, he said, though data teams, developers and even IT teams are asking for a level of flexibility when it comes to building their pipelines.
"Ascend's flex-code enables a multi-layer approach to pipeline building, with the ultimate goal of delivering the highest level of efficiency depending on the level of granularity you're looking to utilize," Leone said.
Why data ingestion needs to get easier
Making it easier to connect to different data sources for analysis and data engineering is not a new goal.
According to Sean Knapp, founder and CEO of Ascend.io, there has been a significant amount of work done in the data warehouse world to create easy-to-use data connectors. That isn't the case for data lakes, however, and it's still challenging and time-consuming for many data engineers to connect their Spark workloads to various services and APIs.
The Ascend flex-code data connector framework is enabled by a user interface to build and configure data connectors.
"What we announced are more than 40 new connectors with no-code interfaces, built using the flex-code data connectors framework," Knapp said. "While users were previously able to create their own custom connectors, the flex-code foundation means they can now turn these into fully reusable connectors that look and feel like native connectors, even with their own no-code interfaces."
How flex-code data connectors work to enable data pipelines
Knapp said the technology behind the flex-code data connectors is proprietary to Ascend.io. Ascend users can leverage the framework to create their own data connectors that can be shared, however.
Ascend's flex-code enables a multi-layer approach to pipeline building, with the ultimate goal of delivering the highest level of efficiency depending on the level of granularity you're looking to utilize.
Mike LeoneSenior analyst, Enterprise Strategy Group
The new flex-code data connectors work by simplifying connector implementation into a few Lambda-style functions that Ascend.io or developers implement. A lambda function is a block of functionality within a code block that can be shared and reused.
Once the parameters of the function are defined, Ascend.io includes the new connector in its suite of available connectors, and dynamically renders no-code user interfaces for creating, browsing and configuring instances of that connector. Knapp explained that under the hood, Ascend converts the Lambda-style functions to Spark jobs that return Spark DataFrames. Those DataFrames can be parallelized and can then be plugged into the rest of the Ascend platform, which includes automatic data profiling, intelligent persistence and incremental data propagation features.
The data connectors are a capability in Ascend Ingest, a feature in the standard Ascend Unified Data Engineering Platform. Knapp said Ascend Ingest already provides users with automatic data change detection, data profiling and reformatting capabilities.
Knapp said he expects more development on the flex-code idea within his company, expanding the concept into other areas.
"Keep an eye out for us expanding our flex-code functionality beyond connectors and into transformation logic itself, enabling powerful, end-to-end no-code applications," he said.
