metamorworks - stock.adobe.com
Vast Data targets secure, speedy agentic AI development
The vendor is furthering its expansion into AI development with features such as vector search and event streaming aimed at enabling users to build and maintain agents.
Vast Data on Wednesday unveiled new features, including real-time vector search and fine-grained access controls, designed to enable users to securely train and update agentic AI applications.
In addition, the vendor introduced event streaming capabilities to deliver data to agentic AI tools in real time. Each of the three features is scheduled for general availability before the end of June.
Many enterprises have rapidly increased their investments in AI development in the two-plus years since OpenAI's November 2022 launch of ChatGPT represented a significant improvement in generative AI technology. However, enterprises need to combine models with proprietary data to enable generative AI models to understand their unique operations.
In October, Vast Data unveiled its InsightEngine, a suite that includes an integration with the microservices-based Nvidia NIM AI development platform. The suite provides users with the tools to manage the AI lifecycle from data ingestion through training and deployment.
Given the surging interest in AI development, Vast Data's new capabilities are significant for the vendor's customers, according to Kevin Petrie, an analyst at BARC U.S.
"Vast Data clearly understands that AI adopters need diverse data to generate valuable outputs and achieve results," he said.
Structured tables remain the favored input for AI models, he continued. But unstructured data such as text and images that can be accessed using vectors are now being used to inform AI tools by more than half of all respondents to a BARC survey while almost half are using real-time data.
"It makes sense to have a consolidated platform that can query tables, search and retrieve vectors, and process real-time events in support of multi-model AI applications," Petrie said.
Based in New York City, Vast Data is a data platform vendor that, like many data management vendors, is adding tools that enable users to develop AI tools.
Enabling AI development
When enterprises first increased their investments in generative AI after the launch of ChatGPT, development focused mainly on building assistants and chatbots that enable workers to query and analyze data using natural language. Such tools make business intelligence available to non-technical workers after decades of analytics requiring coding knowledge that made it the exclusive domain of experts.
Now, investments include agents, generative AI applications capable of autonomous behavior, such as insight generation and automating repetitive tasks to relieve humans of time-consuming, mundane work.
Whether assistants that make employees better informed or agents that make them more efficient, accuracy and security are pivotal.
Accuracy requires large amounts of high-quality data -- including unstructured data that is estimated to account for over 80% of all data -- to reduce the occurrence of AI hallucinations. Security requires governance to ensure regulatory compliance and that the AI tools and the data used to train them are properly used.
Vast Data's new capabilities address both accuracy and security.
Vectors are numerical representations of data automatically assigned by algorithms and are a key way to give structure to unstructured data to make it discoverable. In addition, they enable similarity searches so relevant data can be moved in batches and other forms into data pipelines.
Though not new, vector search and storage have become essential parts of the AI development lifecycle. As a result, many data management vendors expanding into AI development have added vector capabilities. Vector database specialists include Pinecone and Qdrant, while tech giants AWS and Oracle are among those that have added vector capabilities to data management platforms.
Vast Data's vector search and retrieval capabilities within the Vast DataBase allow users to search vector spaces in constant time -- each search takes the same amount of time irrespective of size -- so they can scale searches for agentic workflows. In addition, AI-powered similarity searches enable users to operationalize real-time data automatically.
New serverless triggers and functions in the Vast DataEngine enable users to create real-time workflows without necessitating extract, transform and load tools so event data can provide existing AI applications with new context.
Lastly, Vast Data's fine-grained access controls include advanced row- and column-level permissions to better ensure the security and compliance of analytics and AI workloads.
Collectively, the new features aim to enable customers to securely perform RAG operations in real time at any scale to feed AI applications, according to Aaron Chaisson, Vast Data's vice president of product and solutions marketing.
Regarding the impetus for adding the features, observations of customer needs played a part, he continued.
"As we see more companies become AI-driven, including our customers, we have witnessed the need for fast access to real-time data … while maintaining rigid security, encryption, and governance controls across all data sources," Chaisson said.
Petrie noted that while other data management platforms, such as Google Spanner and vector specialist Vespa, similarly enable users to combine multiple data types and deploy different retrieval methods, Vast's performance and scalability compare well against peers. In addition, enabling workloads in cloud, hybrid and on-premise environments is important for independent vendors such as Vast when competing with tech giants for customers.
"Our surveys show strong and persistent interest in running AI workloads on premises or in hybrid environments," Petrie said. "Independent platforms such as Vast can address this market opportunity while still integrating with cloud-focused hyperscalers."
Next steps
With the new vector search and retrieval, event streaming and security features moving toward general availability, Vast Data has a broad product development roadmap for the remainder of 2025.
Chaisson did not mention specific areas of focus, but said the vendor plans to add new capabilities across its entire platform, which includes DataStore and Data Space in addition to DataBase and DataEngine.
Petrie, meanwhile, suggested that Vast develop an integration with Apache Iceberg, a popular format for storing open tables. Even Databricks, whose Delta Lake is Iceberg's primary competitor, now supports Iceberg tables.
"Given the popularity of Iceberg, I would recommend finding … ways to coexist and integrate with this open table format," Petrie said. "That is a significant addressable market [Vast] will want to penetrate."
Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than 25 years of experience. He covers analytics and data management.
