Cohere tackles some generative AI challenges with Command-R
The startup's new large language model aims to address problems with factual accuracies in generative AI models. It also focuses on language problems, and cloud and API challenges.
AI startup Cohere's new large language model seeks to address some challenges enterprises face with generative AI technology.
Cohere on March 11 introduced Command-R, a generative LLM optimized for long-context tasks such as retrieval-augmented generation and using external APIs and tools. Retrieval-augmented generation (RAG) is a technique used to improve the accuracy and reliability of generative AI models.
The new LLM is available through Cohere's hosted API. It will soon be available on major cloud platforms, according to the independent AI startup, which has a partnership with and investment from tech giant Oracle.
Large context window and accuracy
The LLM has a 128k context window and is available in 10 major languages, including English, French, Spanish, Korean, Japanese and Chinese. A context window is the textual or video range that an LLM can process.
Command-R comes as AI vendors such as Google are introducing models with context windows as large as 1 million tokens. However, large context windows could lead to a higher probability of inaccurate outputs by the LLM.
Cohere's Command-R aims to address the challenge of factual inaccuracies surrounding RAG so that as enterprises input large documents, they're likely to get accurate answers in what the LLM generates, Forrester Research analyst Rowan Curran said.
For the enterprise, having an offering that's designed around what is emerging as the standard architecture for the current generation of large language model-powered applications is an appealing thing.
Rowan CurranAnalyst, Forrester Research
"The main thing that is approached with this release is being able to ensure that you can give very large amounts of information to these large context windows and be more assured that it's going to remember everything within that context window when it produces the generation on the other side," Curran said. "For the enterprise, having an offering that's designed around what is emerging as the standard architecture for the current generation of large language model-powered applications is an appealing thing."
A holistic approach
Futurum Research analyst Paul Nashawaty said Cohere also takes what he calls "a holistic approach" with Command-R.
Usually, LLM providers introduce models using specific databases or data warehouses. However, Command-R will be available across major cloud providers as well as on premises.
"That's not just in one data set, but it's across all the data sets within your ecosystem," Nashawaty said. "So that's really huge."
For developers, this allows them to connect to newer or existing data sources, he added. For organizations with existing data sets, Command-R enables them to use older as well as modernized environments.
Another differentiator is Command-R's pricing model, Nashawaty said. It costs $0.50 per million input tokens and $1.50 per million output tokens. The pricing model allows for API consumption based on usage, which matters when connecting to the cloud, he noted.
"The number of API calls can increase your cost," Nashawaty said. "By having fewer API calls, you can reduce your costs on the cloud."
Some limitations
Command-R's language capabilities are a step in the right direction, but more can be done, Curran said, noting that the LLM is not available in any languages from India or Southeast Asia.
"That is one of the interesting linguistic gaps in a lot of these language models overall, and it continues to persist here," he said.
Moreover, since generative AI is still a new market, the promise of scalability with Command-R will need to be vetted, Nashawaty said.
"They do tend to focus on scalability as a key differentiator," he said. "I just question about the specific use cases. They can search millions or billions of documents, which is great. I'd like to see that in a real-world scenario."
Command-R's model weights are now available on Hugging Face for research and evaluation.
Esther Ajao is a TechTarget Editorial news writer and podcast host covering artificial intelligence software and systems.
