SingleStore unveils features aimed at enabling real-time AI

Vector search

Vector search is an important enabler of generative AI.

Generative AI models are larger than traditional AI models and need to be trained on huge amounts of data to reduce the potential for incorrect responses -- termed AI hallucinations -- that generative AI models automatically produce when they don't have the right data to respond to a query.

As a result, unlike traditional AI models that may be trained on only structured data, generative AI models are generally trained using both structured data, such as financial records and transaction logs, and unstructured data, such as text, audios and images.

That additional data gives the model more data to draw on, therefore reducing its inclination to make up an answer should it not have the right data.

However, training a massive generative AI model with the most pertinent data is difficult, which is why vector search is gaining importance as more organizations look to build customized large language models (LLMs).

Vector searches enable data engineers and other model developers to discover data based on its similarity to other data, which reduces the time and effort it takes to search for and find all the relevant data for a particular generative AI model.

SingleStore has offered exact nearest neighbor vector searches, which enable customers to find exact matches within their data, since 2017.

Now the vendor is adding approximate nearest neighbor (ANN) search, which not only expands the parameters of a given search and its potential results but also substantially increases the performance speed of searches as well as the accuracy of those searches, SingleStore said in a press release.

That increased speed, meanwhile, is significant for those developing generative AI models, typically using text-based JSON documents, according to Doug Henschen, an analyst at Constellation Research.

"Approximate nearest neighbor is much faster than exact nearest neighbor search. And it's more appropriate to the semi-structured … data likely to be associated with generative AI," he said.

Some SingleStore users may find the new compute layer a more significant addition, Henschen continued. But for those concerned largely with generative AI, ANN search in concert with other vector search improvements is important.

In addition to ANN search, SingleStore aims to improve the speed of vector searches by providing the following:

• New algorithms for vector indexing that reduce the time it takes to build indexes within vector databases.
• Combined vector-based semantic search with exact keyword search to speed retrieval-augmented generation time to foster real-time analysis.
• A simple interface that enables customers to use SQL or MongoDB's query language rather than query languages specific to a given vector database.

"The standout announcement is really in the eye of the beholder," Henschen said. "For those entirely focused on generative AI innovation, [it is] the addition of approximate nearest neighbor vector search for faster performance … along with the combination of semantic and full-text search."

Meanwhile, the impetus for developing the new vector search capabilities came from a combination of customer requests and SingleStore on its own recognizing a need, according to Madhukar Kumar, the vendor's chief marketing officer.

"Every customer we talk to is thinking about [building] LLMs or has already started the process," he said.

But customers quickly realize that developing LLMs takes significant time, effort, and compute power, which leads to significant costs. Speed and efficiency reduce those costs.

"Cost and efficiency become a huge part of creating vectors and searching them in real time," Kumar said. "[Real time] was our vision going forward. But with LLMs, that has become front and center."

More capabilities

Beyond improved vector search capabilities, SingleStore unveiled Aura, a new compute layer on the vendor's database platform that lets users deploy GPUs and CPUs for AI and machine learning workloads near where their data resides.

The compute layer enables developers to build LLMs and machine learning models without having to move data outside the SingleStore platform, which not only saves time and effort by eliminating extract, transform and load workloads but also keeps data in a governed environment to ensure data security.

When organizations are forced to export data to combine it with LLM capabilities, they risk exposing their data to breaches. But when they can instead import the LLM capabilities their data within a database, the risk of exposure is reduced.

In addition to Aura, the vendor launched SingleStore Notebooks, which are web-based Jupyter notebooks where developers can collaborate to build models and other data science assets using SQL or Python code.

Included in Notebooks is Spaces, a set of prebuilt notebooks that can be used to help customers develop generative AI applications, recommendation engines and real-time workflows.

Meanwhile, included in Aura and connected to Notebooks is Job Service, which enables customers to schedule SQL and Python jobs -- including automating repeated tasks -- from within SingleStore Notebooks.

Related to both Aura and Notebooks is Radian Replication Service, a set of services that lets users replicate data from various sources into SingleStore.

Aura is in limited preview while Job Service is in private preview. Both Notebooks and Radian Replication Service are generally available.

Together, Aura and its related features will prove even more significant for SingleStore customers than the improved vector search capabilities, according to Henschen.

"To me, the most important collection of new capabilities is the SingleStore Aura 'intelligence layer' and related Notebook, Job Service and Radian data replication service announcements," he said. "All of these things will work together to enable customers to do more with their data without copying it or moving it out of SingleStore."

A new free pricing tier is also now available. SingleStore previously offered a free trial with $600 of credits. Now, however, the vendor's SingleStoreDB Free level provides users with a sandbox environment for hosting data and testing workloads.

SingleStore otherwise is available in three tiers, with Standard starting at 80 cents per hour; Premium at $1.60 per hour, with support for mission critical workloads; and Dedicated for an unlisted amount for enterprises with significant security concerns.

Finally, SingleStore unveiled new and improved integrations that enable developers to build generative AI models using the platform of their choice.

Among them are an extension of an existing integration with Google that now includes Vertex AI, a one-click feature so users can deploy LLMs on AWS using Amazon SageMaker; two integrations with IBM's Watsonx; and numerous connectors to open source platforms, including OpenAI and Vercel.

While the integrations, vector search capabilities and new compute layer are seemingly separate features, they work together to move models from development through deployment, according to Kumar.

Although vector search may not seem related to an integration with SageMaker, vector search is what helps begin the process of developing an AI model that eventually gets put into production.

"Hybrid search that includes vectors, a layer of compute that makes it easier and faster to run workloads next to the data, and the ability to run in all the ecosystems is our overall platform," he said.

Down the road

As SingleStore continues to invest in product development, AI will remain a priority, according to Kumar.

Approximate nearest neighbor is much faster than exact nearest neighbor search. And it's more appropriate to the semi-structured -- typically JSON document -- data likely to be associated with generative AI.

In particular, the vendor's focus will be on enabling customers to use their own data to develop models that inform decisions and making the data informing those models as close to real time as possible.

"AI is not useful -- in fact, it's gimmicky -- unless it is aware of your data," Kumar said. "What is going to be critical is the ability to query and curate that data in split seconds. Our vision is to give companies everything they need to contextualize that data in real time."

He added the customers have historically used SingleStore for transactions and analytics. The next step for the vendor will be to add contextualization capabilities.

Henschen, meanwhile, noted that SingleStore would be wise to continue investing in marketing. In particular, increasing awareness of its cloud-based capabilities is key.

"Anything SingleStore can do to promote broader recognition, adoption and use of the SingleStore Helios cloud service will be to the long-term benefit of the vendor and its core product," Henschen said. "Scale is everything, whether talking about numbers of customers and users, economies of cloud resources in use or funds for future research and development."

Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.