The startup develops language processing units and sells them as a service. While the vendor must deal with market giant Nvidia, it has an opportunity to stay competitive.
AI inference vendor Groq revealed on Monday that it raised $640 million in a series D funding round, bringing its valuation to $2.8 billion.
The funding round was led by BlackRock Private Equity Partners, with a contribution from Neuberger Berman andType One Venture
The Groq funding development emerges amid a market dominated by AI hardware and software giant Nvidia.
Challenging Nvidia
However, the notably large chunk of capital Groq has raised appears to be a vote of confidence for the AI inference vendor, said Mark Beccue, an analyst with TechTarget's Enterprise Strategy Group.
"We haven't seen that kind of money from a chip company lately," Beccue said.
Groq was founded in 2016 by former Google engineer Jonathan Ross. Ross designed and implemented the core foundation of Google's tensor processing unit (TPU) chip.
As a vendor, Groq develops a language processing unit and sells it as a service. The startup plans to deploy over 108,000 LPUs by the end of first quarter 2025.
Despite having an architecture like Google's TPUs and targeting the inference market, it might be an exaggeration to say that Groq is going head-to-head against Nvidia, said Chirag Dekate, a Gartner analyst.
"Nvidia has deep entrenchment -- everything from silicon through software systems," he said.
However, there is a supply shortage in the AI inference market for GPUs, creating opportunities for Nvidia's competitors and other startups like Groq.
"The investing community is sensing that there's a supply-demand imbalance around immediate GPUs, and the market is rewarding any alternative that can help plug even part of the gap," Dekate said.
The $640 million Series D round for Groq is a fraction of the revenue that Nvidia makes in a quarter. The funds are, in some ways, hedge plays by investors, he added.
Moreover, many enterprise AI customers are likely not going to turn to specialized providers if they have already invested millions into their existing cloud providers, such as AWS, Microsoft and Google, Dekate continued.
"An episodic utilization of inferencing may make sense, but I don't think this would be a strategic play for enterprise entities," he said. "This is more for model innovators that can access a lot of these resources at low cost."
A push for Nvidia
However, Groq can be the push Nvidia needs to be challenged, Futurum Group analyst Olivier Blanchard said.
"One of the dangers of having extremely dominant companies in a particular industry like this is that if they're not really challenged a whole lot, you might end up with a very linear innovation [and a] very predictable innovation track," Blanchard said.
We haven't seen that kind of money from a chip company lately.
Mark BeccueAnalyst, Enterprise Strategy Group
An evenly paced innovation track has benefits, including being easily understood and providing the opportunity for budgeting product roadmap. But a predictable innovation track also could result in a longer time to get good products to market.
"If nobody's really challenging Nvidia -- or any company, but Nvidia in this case -- to really innovate as fast as it can, it's going to innovate comfortably as opposed to pushing its own limits," Blanchard said.
Groq might not challenge Nvidia immediately, but it could force the much bigger vendor to push ahead more to keep its top spot, he said.
More AI companies in the market also tends to diversify the supply chain, leading to more capacity, Blanchard added.
Opportunities for Groq
For Groq, it might be better to use the new funding to build a complete stack and expand into the training market to provide an alternative to Nvidia in that area as well, Dekate said.
Meanwhile, the overall generative AI market creates an opportunity for Groq, Beccue said.
Generative AI systems have massive workloads that create openings for vendors in training and inferencing. This has led to some market success for chips like Intel's Gaudi for inferencing and AMD's latest MI325X accelerator.
While Groq might be in a similar situation as Intel and AMD in terms or producing realistic alternative to Nvidia's chips, the cost or production will be challenging for the startup, Beccue said.
"It's a capital-intensive business," he said. "This is not an industry or a product that's really built to favor a startup."
The U.S. government has been encouraging chip vendors with programs like the CHIPS Act. Intel, Micron and Samsung have all received funding through the CHIPS Act.
Esther Ajao is a TechTarget Editorial news writer and podcast host covering artificial intelligence software and systems.
