Next Pathway gives data warehouses a route to the cloud

Next Pathway's Crawler360 aims to help customers take the first step to leaving legacy data warehouses and data lakes behind.

The newly introduced Crawler360 tool scans legacy data warehouse and data lake environments to identify which workloads can be moved to the cloud, how to move them while being mindful of business impact and service-level agreements, and how much time and money the migration will take. Crawler360 looks at data pipelines, database applications and BI tools to build an understanding of all the data dependencies within the environment, which allows it to identify redundancies and figure out what makes sense to migrate.

The actual migration is carried out by Next Pathway's Shift tools: Analyzer, Jet Interpreter, Translator and Tester. This suite of proprietary technology automatically translates code and scripts from the legacy data warehouse environment over to the new cloud environment and includes a validation step to ensure that everything works the same way after migration. Next Pathway's list of supported legacy source systems includes Teradata, Netezza, Informatica, Pivotal Greenplum and Talend. Supported cloud targets include Snowflake, Amazon Redshift, Azure Synapse and Google BigQuery. Once the migration is complete, Crawler360 handles cut-over to ensure applications that consume the data are automatically pointed to the right place.

Other data migration vendors such as Datadobi, Komprise and StrongBox can move data between targets, but Next Pathway stands out by translating data dependencies over to the new target. Normally, customers looking to move data lakes or warehouses would do it manually or through global system integrators (GSIs). In some cases, such as with Google BigQuery, the cloud data warehouse provider may offer a migration service.

Vinay Mathur, chief strategy officer at Next Pathway, said simply overcoming the first step of figuring out where to begin is a challenge because customers' end users would dump data everywhere, leading to lots of duplicate and useless data. Customers would first need to untangle a mess of data lineage and "spaghetti infrastructure," which Crawler360 was designed to address.

Cloud adoption has been steadily growing and got a boost when COVID-19 hit, but Mathur said most enterprise workloads are still on-premises. He said most workloads on the cloud now are simple applications with few data dependencies, but enterprises are running out of those and looking to move more complex and "riskier" workloads. Data warehouses are a bottleneck in migration because converting them to make them runnable in the cloud takes more time and labor than simply putting data in a new place. On top of that, customers realize their warehouses have data management problems and would like to clean them up before moving them.

People now aren't just lifting and shifting to the cloud. They want to lift and modernize -- a more strategic migration so that you don't persist your on-prem problems into the cloud.

"People now aren't just lifting and shifting to the cloud. They want to lift and modernize -- a more strategic migration so that you don't persist your on-prem problems into the cloud," Mathur said.

Mathur added that although Next Pathway competes with GSIs, it also partners with some of them because its automatic translation capabilities help them do their jobs.

Jason Bloomberg, president at analyst and advisory firm Intellyx, said he knows of no direct third-party competitors to Next Pathway, and its biggest competition comes from customers trying to move their data warehouses to the cloud manually. Next Pathway overcomes the difficult challenge of translating interactions within data warehouses to the new cloud environment and focuses on the software that interacts with the data rather than the data itself. This is unique for a third-party tool, according to Bloomberg.

Bloomberg stressed that Next Pathway isn't something a customer can simply buy and deploy themselves. Much like with a GSI, customers would have to work closely with Next Pathway to plan the migration. Mathur said this planning period takes about eight weeks. The advantage of using Crawler360 and Shift is that it's faster than a manual migration due to their automatic scanning and translating capabilities.

Bloomberg said Next Pathway has done a good job finding its role, as there is a lot of demand among enterprises to get off their old data warehouse platforms and take advantage of modern technology. Even though this sort of migration is "one-and-done," Bloomberg claimed there is no shortage of customers who need this. He also pointed out that there are always new target environments to move to, or regulations may change and make current modern environments insufficient for maintaining compliance.

"There are still plenty of companies looking to leave legacy data warehouses -- plenty of business to sustain Next Pathway," Bloomberg said.