Meta intros its biggest open source AI model: Llama 3.1 405B

Against the current

The introduction of the large model is a reversal from a recent trend in the AI marketplace toward small language models.

"It's interesting that it really kind of goes against the grain of all the trending we've seen with language models," said Mark Beccue, an analyst with TechTarget's Enterprise Strategy Group.

A possible reason Meta chose to do this is because Llama 3.1 405B is the first customizable open source LLM, Beccue said.

"Now there's an option for a lot of these companies that are leaning into open source, and there's lots of them, to have a really big model to choose to go after and doing what they want with it," he said.

However, creating a large model like 405B comes at a steep cost.

Meta said that to train Llama 3.1 405B, it used more than 16,000 of Nvidia's H100 GPUs. Those AI chips cost between $25,000 to $40,000 each depending on the configuration, meaning Meta spent up to $640 million to train the new model.

Therefore, Llama 3.1 405B might be too costly for some enterprises to deploy and maintain, said Futurum Group analyst Paul Nashawaty.

"The 405 billion-parameter model demands immense computational resources, including high-performance GPUs and substantial storage," he said in a statement to media outlets. "This translates to significant upfront costs for hardware, as well as ongoing expenses for electricity and cooling."

Thus, for small enterprises, generative AI tools already available on the major cloud platforms might be less expensive, he continued.

Valuable for enterprises

However, a large open source model unlocks value for enterprises, said William McKeon-White, an analyst with Forrester.

"Organizations are figuring out that GenAI is still kind of hard," McKeon-White said.

"Having an open source model or something that you're not paying per transaction for can go a long way for organizations that are looking to use these models in a very tailored fashion," McKeon-White continued.

 Moreover, models like these provide more complex reasoning, and many of the applications that organizations envision using the models for, such as fraud detection and medical diagnosis will, require complex reasoning, he added.

However, controversy surrounds what Meta considers open source despite Meta CEO Mark Zuckerberg's blog post, also released on Tuesday, detailing his company's commitment to open source. Some argue that the Llama models are not fully open source because Meta has not released its training data.

"It's open source, to the extent that you can custom train it, you can customize the model, but you still don't know what the data sources are," Beccue said. "To me, that leads to this inherent problem we have with these models, which is, you can't trace the accuracy."

Moreover, it's unclear if Llama 3.1 is more accurate than Llama 3, Beccue added.

Regardless, Meta differs from Google and OpenAI in that it is willing to make its biggest model customizable so others can build on it, said Arun Chandrasekaran, an analyst with Gartner.

"The models are still quite invaluable for enterprises, and you still get access to model weights, which I think is quite important for enterprise clients," Chandrasekaran said.

Scaling safety

Other than the large parameter size of Llama 3.1 405B, Meta revealed that it's also scaling its AI safety capabilities. The social media giant introduced two new safety tools.

Llama Guard 3 is an input and output moderation model that helps developers detect violating content.

Prompt Guard is another tool that helps developers respond to prompt injection and jailbreak inputs.

Prompt injections use data from untrusted sources to try to make a model perform in unintended ways. Jailbreaks are instructions that try to override a model’s safety and security features.

Meta also revealed that it conducted both human and AI-enabled red team testing to understand how its models perform against various adversarial actors and in different situations.

"They are starting to take safety a little bit more seriously, and they're starting to put some efforts around there," Chandrasekaran said.

Monetization and other news

One substantial challenge for Meta and other vendors that continue to innovate is how to make money off their products, Chandrasekaran said.

"We are continuing to see innovation in this space, but at the same time, I believe that more and more companies are at this moment where they're starting to recognize that they're building all of these very powerful products, but they're all struggling with monetization as well," he said.

Llama 3.1 is available now on AWS.

Scale AI also revealed that it has partnered with Meta to help businesses customize, evaluate, and release Llama 3.1.

AI hardware and software vendor Nvidia also revealed that enterprises can use Nvidia AI Foundry to customize open models from Nvidia and third party open models, including Llama 3.1.

Meta also revealed that its AI assistant, Meta AI, is now available in seven new languages and more countries around the world.

Meta AI users now have the option to use Llama 3.1 405B on WhatsApp and meta.ai.

The assistant is also now more creative, according to Meta, with new "imagine me" prompts that let users create images with different "imagine me" prompts.

Esther Ajao is a TechTarget Editorial news writer and podcast host covering artificial intelligence software and systems.