Rockset targets cost control with latest database update

New capabilities

Vector search and storage have become critical capabilities over the past year due to how vectors can inform generative AI models.

Vectors are a means of giving structure to unstructured data, such as text and audio files.

Unlike structured data, such as financial records and point-of-sale transactions, that is easily operationalized, unstructured data often gets uploaded into a data lake, where it becomes essentially undiscoverable amid potentially billions of other unstructured and undiscoverable data points.

However, by using algorithms to assign unstructured data a numerical value -- a vector -- unstructured data points become searchable and can be used to help inform models and applications, including generative AI.

Beyond giving unstructured data a value, vectors have become critical components of generative AI modeling because their characteristics enable similarity searches that can be used to feed the retrieval-augmented generation pipelines that train large language models (LLMs) and other applications.

Vector search and storage, however, require enormous amounts of compute power and are thus expensive.

Therefore, as enterprises perform an increasing number of vector searches to train LLMs to understand their own proprietary data. As organizations assign vectors to increasing amounts of unstructured data as it gets captured and ingested into databases such as Rockset and MongoDB, computing costs are increasing beyond expectations.

"Vectors are ridiculously expensive," Venkataramani said. "So efficiency matters and reliability matters."

He noted that vector searches without sufficient compute power will fail. The searches will take days and possibly never even finish.

Such searches, therefore, require sophisticated compute indexes, which then must be combined with other data indexes -- adding up to the use of massive amounts of data -- to result in relevant query responses.

"It needs a lot of memory, and it needs a lot of RAM," Venkataramani said. "[As a result], the fundamental scalability limits are related to the cost effectiveness and compute efficiency of the back end [of the database]."

To address cost control, Rockset's latest update therefore aims to help enterprises by improving the efficiency of its database with the new features resulting in up to a 30% improvement in price performance.

The new autoscaling capabilities enable customers to scale up compute power when needed to carry out workloads and power down when workloads are reduced or not running. The new memory-to-CPU ratios allow users to decouple different compute types from each other to make indexing and querying more efficient.

Perhaps just as important as the update, however, is the strategic investment from HPP, according to Catanzano.

"The investment by HPP [gives Rockset] financial support, a big brother and potentially a future acquirer," he said.

Catanzano added that Rocket is a niche vendor compared to tech giants, such as Oracle, that offer widespread database capabilities. But its focus on real-time analytics makes it attractive no matter the size of its competitors.

"They can play alongside many other databases and data lakes that focus on AI and analytics workloads," he said. "They [have] vector search, and others are trying to catch up with that capability now. I especially like the real-time aspect [of Rockset] since time to insights is critical and a competitive edge for businesses."

The impetus for the new capabilities, meanwhile, came from conversations with customers, according to Venkataramani.

He noted that developers want to experiment with their organization's data and come up with new applications based on those experiments. Holding them back, however, are budgetary restrictions that limit the extent of their experimentation.

While improved cost efficiency won't allow developers to experiment without cost limits, it will at least allow them to do more experimentation for the same cost.

"Application developers always … have hundreds of ideas but don't have the budget," Venkataramani said. "They have the ideas, and [with more efficient tools] can do more with less."

Next steps

Efficiency will continue to be a focal point of Rockset's product development plans, according to Venkataramani.

In addition, expanding to new clouds is an emphasis for Rockset.

Currently, the vendor's database tools are available only on AWS. Over the next year, Venkataramani said Rockset plans to become available on Microsoft Azure.

Vectors are ridiculously expensive. So efficiency matters and reliability matters.

He noted that many of the vendor's customers store their data on Azure. In addition, Microsoft has been an aggressive developer of tools such as Azure AI Studio that enable users to build their own generative AI applications.

Once Rockset is available on Azure, Google will be next, Venkataramani continued.

"We're going to basically follow the market in terms of what prospects are telling us," he said. "As of now, we can clearly see that Azure needs to be a priority. Google is a close second after that. Eventually, we will be in every cloud. It's just a matter of sequencing it in the right order to maximize the impact it has on our business."

Meanwhile, Catanzano said that Rockset would be wise to continue focusing on enabling real-time AI and analytics.

He noted that by concentrating on cost control along with real-time capabilities, the vendor has an opportunity to attract new users.

"[They should] drive the time to insights down even further," Catanzano said. "They can address the lower end of the market to give the smaller companies a chance to compete [and] are doing this with the low cost they mention."

Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.