Meltano spins out from GitLab to advance DataOps platform

Getting data out of one platform and into another is a challenge organizations approach with a variety of tools.

For collaborative development platform GitLab, the answer to its data integration challenge was to build its own extract, load and transform (ELT) platform, known as Meltano. On June 30, GitLab said it was spinning Meltano out as its own independent company. The new company is being launched with $4.2 million in seed funding, led by venture capital firm GV, formerly known as Google Ventures.

Meltano is built on a series of open source technologies, including the Singer project for data connectors and dbt for data transformation. The goal for Meltano is to build out a data operations (DataOps) platform that can help organizations deploy data pipelines to use data for business intelligence and analytics. Currently, Meltano is all open source, but the plan as a vendor company is to build out commercial services, including a managed cloud service in the future.

In this Q&A, Douwe Maan, founder and CEO of Meltano, outlines the new company's approach and where Meltano is headed.

What is Meltano and its vision for DataOps?

Douwe Maan: From day one, we had this idea of building a kind of end-to-end solution for the entire DataOps lifecycle. So, we identified what we thought are the different steps in that lifecycle and we basically made an acronym out of them. So, Meltano stands for model, extract, load, transform, analyze, notebook and orchestrate.

Meltano started out because the internal data team at GitLab had a need for a data stack that aligned more with its values and principles. In 2018 when we started Meltano, we found that most data tools were kind of stuck five years in the past compared to the advances that the software development space has made. So, we decided to build something ourselves.

Douwe Maan

By using open source technology and providing, essentially, the glue between different components, we built a platform that allows different open source technologies to work together to build something that becomes better than the sum of its parts. Meltano is an open source platform that aims to bring the entire data lifecycle into the DataOps way of thinking about data projects.

Why spin Meltano out as a standalone company now?

Maan: Meltano was not intended to be another product line for GitLab; it was really an internal tool for the data team. But we realized really quickly that there are a lot of companies that have the same needs as GitLab for a data stack.

In the last year, we have seen significant growth and adoption of Meltano. We realized that, in order to keep growing the project, it made more sense to spin out. GitLab wants to be able to stay focused on its single platform for the entire DevOps lifecycle. By spinning Meltano out, it keeps both companies focused.

We identified what we thought are the different steps in [the DataOps] lifecycle and we basically made an acronym out of them. So, Meltano stands for model, extract, load, transform, analyze, notebook and orchestrate.

How do you define DataOps?

Maan: DataOps is really about taking DevOps principles and practices and applying them to data. That means starting to think of data pipelines more as a type of software engineering.

For me, I'm thinking about allowing teams to be more collaborative and efficient through code review, version control and integration with CI/CD [continuous integration/continuous deployment].

So, when I think of DataOps, it has more to do with DevOps processes and the efficiency of collaboration.

What's next for Meltano?

Maan: We are planning to spend, at least, the remainder of the year really focusing on the product and making it a preferred solution for data teams and software developers tasked with data challenges. We will start integrating with more open source technologies to start showing off the vision we have for Meltano, being that stable foundation for data projects, on top of which data product stacks can be built.

So, our first hires we've made already over the last few weeks; [they] are on the marketing and community management side, as well as a number of additional engineers. The focus is really going to be to attract the community who wants better data tooling that fits into the DataOps way of working.

Editor's note: This interview has been edited for clarity and conciseness.