Mistral's new small AI models target phones, laptops
The models are reflective of the trend toward small language models built to run on edge devices. However, it might be hard for the startup to compete.
AI startup Mistral marked the first anniversary of the release of its open source Mistral 7B model by introducing two new models for the edge.
On Oct. 16, Mistral introduced Ministral 3B and Ministral 8B for on-device computing and at-the-edge use cases. While Mistral is known for its small models, the two Ministral models are the first models for on-edge devices from the AI vendor.
The models can be used to orchestrate agentic workflows and create specialist task workers, according to Mistral. They can be tuned to handle input parsing, route tasks and call APIs using low latency and cost, the vendor added.
The small AI model trend
Mistral's new models reflect the use of AI technology across different devices and the growing interest in tiny AI models.
Mistral follows the trend from large cloud providers such as Microsoft and Google. For example, Google has the Gemma model family, which are all 2 billion and 7 billion parameters, and Microsoft has its small language model family, Phi.
The importance of small models is due to a couple of factors, according to Gartner analyst Arun Chandrasekaran. Smaller models can mean low inferencing costs, he said. It's also easier to run smaller models outside the cloud in on-premises environments and at the edge, on devices.
"That's why the models have been slimming down," Chandrasekaran said. "To cater to the need of distributing in resource-constrained environments."
Industries such as telecom, automotive and manufacturing are environments where smaller models are sought-after.
There is also a push to get the smaller models out to the edge on laptops because of some downsides to centralizing everything in the data center, such as latency, Futurum Group analyst David Nicholson said. While the amount of time to get a response when using ChatGPT is relatively fast right now, users might demand faster outputs.
"The expectation is people will get tired of that quickly," Nicholson said. "The expectation is these small models running on the edge are going to leverage the hardware horsepower that's in these new AI laptops."
Competing with giants
Vendors building the new AI PCs are eager to partner with companies creating AI models for the edge, but small providers such as Mistral might find it challenging to compete with larger AI vendors.
The problem is that they're competing with [vendors] who have a certain amount of captured market share.
David NicholsonAnalyst, Futurum Group
"Typically, the devices come preloaded with whatever their partners' preferred small model is going to be, and that's the problem for Mistral," Nicholson said. "The problem is that they're competing with [vendors] who have a certain amount of captured market share."
For example, if Google plans to sell an AI laptop, it will use its model. The same can be expected for Microsoft.
Mistral must convince a vendor like Qualcomm to use its model, he said.
"In order for them to get this in the hands of people who will use it, it needs to come bundled on devices," Nicholson said. "The real question about whether they'll be successful or not comes down to what their partner ecosystem looks like."
Mistral's previous reputation as an open source vendor might fail to make it appealing to users compared with other open source model vendors like Meta. For one, the new models require a commercial license. Also, many users expect AI models to work easily right out of the box, Nicholson said.
Open source is also difficult to monetize and often requires significant financial backing, Chandrasekaran said. When a model becomes open source, numerous vendors are willing to provide it as a managed service and undercut the original model provider on inference cost, he added.
The Ministral 8B model costs $0.10 per million tokens, both input and output. Ministral 3B costs $0.04 per million tokens, both input and output. The models will be available through cloud providers including Azure AI, Amazon Bedrock and Google Cloud Vertex AI Model Garden.
Esther Ajao is a TechTarget Editorial news writer and podcast host covering artificial intelligence software and systems.
