The compute problem AI startup Modular is targeting
The vendor is looking to enable developers be more efficient in their training and inferencing of machine learning models. It uses a programming language called Mojo.
Less than two years after its founding and a few months after releasing its compute engine and programming language for AI developers, AI startup Modular has now raised a total of $130 million.
The AI software developer on Aug. 24 revealed it had raised $100 million in a funding round led by venture capital firm General Catalyst. The vendor previously raised $30 million in 2022.
Co-founders Chris Lattner, former Google senior director of TensorFlow infrastructure, and former Google product leader Tim Davis, formed Modular in 2022 to improve AI infrastructure and bring modularity.
This idea of boosting developers' productivity interested General Catalyst, according to Deep Nishar, managing director at the venture capital firm.
Modular's mission fits General Catalyst's criteria of what the firm considers when looking at AI companies it wants to invest in, Nishar said.
"We do believe that with the heterogeneity of machine learning compute out there, there has to be a better solution to help developers be a lot more productive in the way they develop both training and inference code as well as performance," he said.
Because Mojo enables developers to transition from research to training within the same development environment, this helps users be more productive, Nishar added.
Modular's computing environment works enables work across various computing environments. This tackles the problem within modern machine learning environments in which enterprises often have heterogeneous AI and machine learning environments and have to inference workloads on CPUs and train on GPUs.
"You don't want your engineers to be developing separately for different types of hardware," Nishar said. "That's very inefficient."
Challenges of compute
What Modular is targeting are the hardware challenges of AI compute, according to Mark Beccue, an analyst with research firm Futurum Group.
There has to be a better solution to help developers be a lot more productive in the way they develop both training and inference code.
Deep NisharManaging director, General Catalyst
"It's currently costly, and because of the lack of purpose-built semiconductors for generative AI, AI computing might be a scarce commodity," he said. "All efforts to develop power and compute efficiencies will mean lower costs for enterprises in their AI efforts."
Modular is one of many AI vendors aiming at the AI compute problem. OctoML, another startup that has raised a similar amount of capital as Modular, is also targeting the same challenge.
OctoML provides a platform for engineering teams to develop machine learning models on any hardware.
Because the AI market is somewhat chaotic amid the rise of new and fast evolving technologies such as generative AI, startups like Modular and OctoML will have to clear up the confusion for enterprises about where they fit in, Beccue said.
"Education and best practices for ways forward will take time," he said. "Can these startups get the audience with enterprises to make their pitch and educate?"
Other participants in the most recent Modular financing round include the vendor's existing investors, Google Ventures, SV Angel, Greylock and Factory.
Esther Ajao is a news writer covering artificial intelligence software and systems.
