Couchbase launches database tools to foster AI development

New capabilities

Couchbase first unveiled Capella Columnar in preview during AWS re:Invent 2023. The service, which is only available on AWS at this point, aims to bring together operational database workloads with real-time analytics in a columnar format that analytics tools can understand.

Many developers, including Couchbase customers, use JSON -- a data interchange format used to move data between web clients and web servers -- when building enterprise applications. JSON, however, can be difficult to use with analytics systems that use different, more rigid formats for storage and analysis, the vendor noted.

As a result, unstructured JSON data often goes unused and lies dormant in a database. Meanwhile, with enterprises now developing generative AI applications that require huge amounts of proprietary data to understand the enterprise's business and respond accurately to business-specific queries, unstructured data is becoming critical.

Unstructured data such as text, images, videos and audio files is estimated to make up more than 80% of all data with the structured data traditionally used to inform analytics just a small part of an enterprise's overall cache of information. Without accessing unstructured data, enterprises don't get a complete view of their business and AI applications trained on their data are more prone to deliver incorrect outputs.

Capella Columnar transforms JSON data so it can be recognized by analytics tools, making previously inaccessible data accessible for informing decisions and training AI models and applications. The feature reduces the cumbersome extract, transform and load (ETL) process by supporting real-time data ingestion, using Capella iQ to automatically write SQL to calculate an analytical metric and writing back the metric to the operational side of Capella where it can be used in an application.

Because Capella Columnar enables operational processing and real-time analytics in one database, its release is an important development for Couchbase users, according to Catanzano.

"The launch of Couchbase Columnar is significant," he said. "It addresses a long-standing challenge of making JSON data useful for analytics, which has traditionally been difficult due to its unstructured nature."

An added benefit could be cost reduction, Catanzano continued, noting that it adds expenses to do operational processing and real-time analytics on separate platforms.

Matt McDonough, Couchbase’s SVP of product and partners, said that while many enterprises are attempting to build more AI applications, including generative AI tools, such applications remain more an idea than a reality.

Tools such as Capella Columnar aim to make it easier to develop AI-powered applications that can be used widely across organizations rather than by just data science teams.

"AI-powered apps have been a relatively abstract concept," McDonough said. "With the availability of these new features in Capella, developers can bring AI apps to life because they're no longer bogged down with rigid systems or complex ETL processes."

Like Capella Columnar, Couchbase Mobile with vector search aims to speed and simplify application development.

Vector search has become a key component of retrieval-augmented generation (RAG) pipelines commonly used to train generative AI models and applications. Vector embeddings are a way to give structure to unstructured data by assigning it a numerical value so it can be searched and used to train models and applications. In addition, vectors enable similarity search that makes data discovery easier than the more limiting keyword search, helping users find enough data to properly inform AI tools.

Following its initial introduction of vector search capabilities in February, Couchbase is now extending those capabilities beyond its traditional database environment to edge devices in a move that stands to benefit customers, according to Henschen.

With Couchbase Lite, a document database that can be embedded into edge devices to enable real-time decisions, developers can build applications using mobile devices that can subsequently be consumed on mobile devices.

"The availability of vector information supports similarity search and improves search accuracy, so it's nice to see in the mobile database as well as the core product," Henschen said.

The impetus for developing both the new mobile feature and Capella Columnar came from Couchbase's recognition that enterprises are struggling to build AI applications, according to McDonough.

Many organizations have complex data systems that include numerous different platforms that don't natively integrate with one another. As a result, the pieces don't always work smoothly with one another leading to data quality issues. In addition, if different departments within organizations use different tools, data often gets isolated.

As Couchbase develops new features, one of its primary goals is to consolidate capabilities in a single database platform.

"For developers to evolve in the age of AI, they have to clean up complex architectures, which means consolidating platforms, eliminating data silos [and] making sure they're working with trustworthy data," McDonough said. "To do this, they need the right resources."

Beyond Capella Columnar and Couchbase Mobile with vector search, Couchbase unveiled a new free tier that will be available starting Sept. 9.

Plans

Toward Couchbase's goal of making it faster and easier to build AI applications, the vendor's roadmap includes improving the developer experience through partnerships and integrations that create an ecosystem and provide key capabilities, according to McDonough.

Catanzano, meanwhile, said Couchbase's focus on enabling users to develop AI tools is appropriate.

In particular, the vendor would be wise to concentrate on helping customers ensure that trusted, high quality data is used to inform models and applications. Given the decision-making speed and scale generative AI enables, it is increasingly critical that the data used to inform generative AI tools is accurate.

"[Couchbase should] continue to innovate around bringing highly trusted, enterprise data into GenAI models in a secure way, using RAG and vector capabilities to help create new and innovative solutions," Catanzano said.

Eric Avidon is a senior news writer for TechTarget Editorial and a journalist with more than 25 years of experience. He covers analytics and data management.