AI vendor SambaNova intros next generation of DataScale

Specialized and accelerated infrastructure

With the next generation of DataScale, SambaNova has improved the model performance of the system by doubling the processor speed and improving the memory capacity of the ecosystem, said Chirag Dekate, an analyst at Gartner.

Like other AI hardware vendors such as Nvidia, Intel and AMD, SambaNova Systems caters to enterprises that need more specialized infrastructure and accelerated production of complex models for applications in the banking, pharmaceutical and energy industries and scientific research.

SambaNova said the U.S. Department of Energy's Argonne National Laboratory is expected to test the updated platform in applications, including using GPT for gene generation, 3D convolution networks for neutrino physics, and prediction of tumor response to single and paired drugs.

"You are starting to see hardware vendors step up and develop capabilities in their processor architectures or accelerator architectures that can actually address larger models at faster timescales," Dekate said.

At the heart of the SN30 design is the capability to get faster compute when integrating hardware and software, said R. "Ray" Wang, analyst and founder at Constellation Research.

Software-defined approach

The DataScale SN30 system lets enterprises take a software-defined approach, Wang continued. "You can bounce from CPU to GPU based on the data types."

You are starting to see hardware vendors step up and develop capabilities in their processor architectures or accelerator architectures that can actually address larger models at faster timescales.

This is what differentiates the SN30 architecture from other AI production-level architectures on the market, Dekate said. Users can reconfigure their processors to meet the demands of their workloads, letting enterprises deliver high-efficiency performance for their neural networks.

"If you are an enterprise today, you no longer just have one processor you need to choose from. You can actually explore and evaluate multiple options," Wang said.

SambaNova also unveiled a new processor for its SN30-based Reconfigurable Dataflow Unit. While the vendor does not sell its chips separately, they are key components of its DataScale systems and Dataflow-as-a-Service, which provides customers pre-trained deep learning models, said Marshall Choy, senior vice president of product at SambaNova.

"This goes into the overall system architecture where we have built our systems to provide massive amounts of memory to run massive models," Choy said, referring to large language models like GPT that require large parameters, computer vision models that call for the highest resolution, and recommendation engines that need embedding tables.

"All of these requirements equate to the need for more physical memory in the system," he said.

Proof of concept

For enterprises that work with large deep learning models and foundation models like GPT, it's important to perform proof of concepts with vendors of GPUs and advanced processor architectures such as SambaNova, Dekate said.

"What you're likely going to discover is for larger, more complex architectures, technology architectures that have higher memory capacity and that are custom tuned for performance will likely deliver better value," he said.

While SambaNova did not reveal the pricing for DataScale SN30, it said it will now offer subscription pricing for DataScale and Dataflow-as-a-Service.

The vendor, based in Palo Alto, Calif., unveiled its updated platform on the second day of the AI Hardware Summit in Santa Clara, Calif.