The partnership aims to address the difficulty of efficiently and inexpensively deploying and managing data models developed in DBT by automating previously manual processes.
Ascend.io on Thursday unveiled a new integration with DBT Labs that aims to enable customers to efficiently and cost-effectively run DBT models with a single command.
Based in Menlo Park, Calif., Ascend.io specializes in data pipeline orchestration and automation.
In June 2023, the vendor integrated with Databricks to help joint users more easily view their data as well as share it to collaborate. Before that, in December 2022, Ascend.io partnered with Snowflake to offer the data cloud vendor's users a free data ingestion tool in a move aimed at attracting new customers.
DBT Labs, whose name stands for Data Build Tool, is based in Philadelphia and offers both open source and for-profit capabilities that enable engineers to transform data.
The vendor began as a fully open source project before developing proprietary software and has aggressively partnered with other vendors such as Alation, ThoughtSpot and Starburst over the past couple of years to expand its reach.
The integration between Ascend.io and DBT Labs -- dubbed Ascend for DBT -- is the first between the vendors. It's designed to enable joint users to automate and optimize control of DBT models and applications without requiring more tools in their data stacks.
The integration
One of the challenges associated with using DBT Core, the open source version of DBT's tools, is that it requires users to either develop or deploy a separate orchestration tool to execute and manage their models, Ascend.io said in a press release.
DBT Core is an environment for developers to build applications and models. It does not, however, include deployment and management capabilities.
The integration between Ascend.io, whose tools are designed to enable users to run and operate advanced data pipelines and applications at scale, and DBT Labs aims to address that challenge.
Using Ascend for DBT, joint users can continue to build models and applications with DBT Core while automatically deploying them to Ascend.io for execution. That execution includes automating the management of data pipelines used to train and update applications and models.
The integration is significant for joint users as it addresses a key challenge in deploying and running DBT models at scale.
Stephen CatanzanoAnalyst, Enterprise Strategy Group
"The integration is significant for joint users as it addresses a key challenge in deploying and running DBT models at scale," said Stephen Catanzano, an analyst at TechTarget's Enterprise Strategy Group. "Prior to this integration, DBT Core users had to rely on a separate orchestration tool to execute their models, introducing added complexity and overhead."
He added that beyond attempting to address the needs of DBT Core users, Ascend.io and DBT Labs are a logical fit. The automation and optimization capabilities provided by Ascend.io should serve their intended purpose, Catanzano said.
"The integration between Ascend.io and DBT Core addresses challenges associated with deploying and managing models built using DBT Core," Catanzano said. "By serving as an automation and optimization controller, Ascend for DBT streamlines the deployment process."
Specifically, he noted that Ascend.io's DataAwareTM intelligence tool automates tasks such as tracking data lineage, identifying code or data changes and generating jobs for efficient pipeline operation.
Sean Knapp
Beyond enabling engineers to manage models and applications after developing them with DBT Core, the integration between Ascend.io and DBT Labs is intended to also lead to cost savings and increased engineering efficiency, according to Sean Knapp, founder and CEO of Ascend.io.
He noted that during the preview process, the integration led to cloud infrastructure savings of up to 30%, while engineering work was reduced by up to 80%.
Deploying and managing data models and applications requires substantial work, including complex concepts such as incremental propagation and data validation. When done manually, such tasks take up the majority of engineers' time.
Ascend for DBT automates much of that work.
"I would say that the impact on the engineering teams is more significant than the cost savings, especially as we see most engineering teams and data teams overloaded with massive backlogs of work," Knapp said. "If we can help them work through those backlogs faster and remove a lot of the [manual] work, it's a win for everybody."
Meanwhile, the impetus for developing the integration between Ascend.io and DBT Labs came largely from customer requests, Knapp continued.
He noted that Ascend.io has customers that like building data models and applications with DBT tools but were having difficulty moving those data products into production and efficiently scaling those data products.
"We try to react and respond to what we see our customers and our prospects needing," Knapp said. "This [integration] is a pretty direct response to them."
Future plans
While Ascend.io continues to develop an ecosystem through integrations with vendors such as DBT Labs, Databricks and Snowflake, Catanzano said the vendor would be wise to focus some of its product development plans on scalability and interoperability.
Ascend.io competes with independent vendors, such as Matillion and Denodo, as well as tech giants that offer data pipeline orchestration capabilities, including AWS with Glue and Microsoft with SQL Server.
"It would be interesting to see further enhancements in the areas of scalability and interoperability," Catanzano said. "Specifically, advancements in scaling capabilities to handle larger data sets and more complex workflows could benefit users dealing with growing data volumes."
Unlike some industry insiders who predict that generative AI will transform data management and analytics, he said he expects large language models to be more of an assistant that can support data experts and business analysts within their workflow.
"I don't think [generative AI] is going to develop pipelines for anyone soon. But I do think it's going to solve the really painful muck of data engineering that most people don't like, such as documenting code, suggesting tests and analyzing code for quality and vulnerability," Knapp said. "That's the kind of stuff I think GenAI will be very good at."
Beyond generative AI, Knapp said that Ascend.io has ambitious plans to develop a technology that will natively connect the disparate pieces that make up the modern data stack.
He noted that Kubernetes served that role for software engineering and cloud infrastructures. But data management and analytics platforms have no single connective capability.
Currently, while software providers are developing integrations between their tools, much of the work piecing together the disparate data ingestion, integration, storage, observability, application development and analytics tools falls on engineers and other IT professionals.
"There is no Kubernetes for data engineering," Knapp said. "Kubernetes … has radically transformed and streamlined what it means to run cloud-based SaaS services. It's helped consolidate a lot of methodology and architecture. We haven't seen that in the data ecosystem yet, so we're making a big investment in trying to figure that out."
Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.
Sean Knapp
