Elastic is updating its flagship platform with a major release that adds enhanced vector search and natural language processing capabilities to improve data query results.
With Elastic 8.0, the data search vendor has updated its namesake platform with new capabilities for searching data across both on-premises and cloud environments.
The Elastic 8.0 release became generally available on Feb. 10, marking the first major version change for the Elastic platform since the release of 7.0 in April 2019.
One of the primary components of Elastic is the Elasticsearch search technology, which uses the open source Apache Lucene data indexing technology.
With Elastic 8.0, search is improved with a feature known as k-nearest neighbor (kNN) search, which can provide more relevant search results than previous versions of Elastic. The update also provides default security settings for both self-managed customers that run on premises and Elastic Cloud users.
Elastic 8.0 includes both incremental improvements as well as some entirely new ones, according to Forrester Research analyst Mike Gualtieri. He sees the kNN search feature in particular as an advanced capability that will help differentiate Elastic.
That [kNN] is the type of search technology you would only have seen previously from Google, [Microsoft] Azure and AWS. The future of enterprise search will include multiple technologies, especially taking advantage of newer AI technologies and techniques.
Mike GualtieriAnalyst, Forrester Research
"That [kNN] is the type of search technology you would only have seen previously from Google, [Microsoft] Azure and AWS," Gualtieri said. "The future of enterprise search will include multiple technologies, especially taking advantage of newer AI technologies and techniques."
Elastic 8.0 improves search with features from Lucene 9.0
From a search perspective, the new update is focused on improving both relevance and performance, said Steve Kearns, vice president of product management at Elastic.
"Elasticsearch is a search engine," Kearns said. "It is really good at taking in documents and really any unstructured data and making it available for search."
Kearns noted that Elasticsearch uses the open source Apache Lucene search technology that was updated to version 9.0 in December 2021. Elastic 8.0 now gets improvements from the new Lucene release, Kearns said, including more efficient memory use and indexing speed.
Elastic 8.0 brings new vector search capabilities
The nearest neighbor approach that improves search relevance has its roots in what Elastic refers to as vector search.
Matt Riley, general manager for enterprise search at Elastic, said search typically has been thought of as providing a user with a set of documents or data that is relevant to a given query. And relevance has largely involved matching keywords in a query to human-readable keywords in the data, where a search technology calculates frequency to determine appropriate response.
Riley noted that vector search works differently, instead transforming human-readable keywords into a mathematical vector. Search results with vector are not focused on keyword relevance, but rather on the nearness of two vectors.
"It requires a fundamentally different matching approach than what has been available previously in Elasticsearch," Riley said.
Determining relevance in a vector search is also done with nearest neighbor matching. With the nearest neighbor approach, a query result will return the closest neighbors, or adjacent vectors, for a specific query.
Natural language processing comes to Elastic 8.0 search
Another key data search improvement in Elastic 8.0 is enhanced natural language processing (NLP) capabilities.
NLP is typically built with machine learning models trained on a set of data to better understand language. In Elastic 8.0, Riley said that trained machine learning models for NLP can now be loaded directly into the platform to improve queries.
