In April 2024, Meta released Llama 3, the latest version of its AI-powered large language models based on a dataset that’s at least 7 times larger than Llama 2.
Initially available in 8B and 70B parameter sizes, Llama 3 already outperformed Llama 2, Google's open-source Gemma, and Anthrophic’s Claude Sonnet at launch. Sonnet has since had an upgrade making it one of the most powerful AI models.
But now, leaks suggest that the much-awaited release of the most powerful Llama 3 models which have been trained on more than 400 billion parameters, might also be close at hand. This is just one of a number of new models Meta is working on, utilizing its hundreds of thousands of Nvidia H100 GPUs.
Efficient yet powerful
📝 WhatsApp beta for Android 2.24.14.7: what's new?WhatsApp is working on a feature to choose the Meta AI Llama model, and it will be available in a future update!https://t.co/fInfKYk8Oo pic.twitter.com/eVqWfJ1wGAJune 26, 2024
In early testing, the instruction-tuned Llama 3 400B scored 86.1 on the MMLU benchmark, which already makes it on par with GPT-4's performance with less than half the parameters.
There’s a lot of technical information to unpack here, so let’s talk about why this really matters.
Simply put, large language models with more parameters always tend to perform better on benchmarks and real-world tasks. But the fact that Llama 3 400B can nearly match GPT-4's MMLU score with under 50% of the parameters, suggests that Meta has made enough advancements in model architecture and training to give OpenAI a serious run for its money.
By achieving equal performance with fewer parameters, Llama 3 400B is likely to be much more efficient than OpenAI’s ChatGPT 4 in terms of computational resources, energy consumption, and cost.
Open-source advantage
Another important reason why people are so excited about Llama 3, is that it has been released under an open license for research and commercial use. Although it isn't yet clear if 400B will be released under that same open license.
If it is released as an open model then these state-of-the-art language capabilities would now be available to researchers and developers for free through multiple cloud platforms and ecosystems, accelerating innovation and enabling more novel applications of the technology.
With the new 400B model packing enough power to rival ChatGPT 4, that puts a lot of power into researcher’s hands. This would allow for more rapid development of advanced language AI applications without relying on expensive proprietary APIs.
What we know so far
Meta AI has been hinting at the release of the 400B model since its original press release about Llama 3 on April 18. “Our largest models are over 400B parameters,” it wrote back then, adding that “over the coming months, we’ll release multiple models with new capabilities including multimodality, the ability to converse in multiple languages, a much longer context window, and stronger overall capabilities.
Since then, the internet has been abuzz with theories and ideas about a possible release date for the 400B models. While the folks at Meta have confirmed that development on Llama 3 400B has already wrapped up, no official release date has been announced as of yet.
Still, WhatsApp Beta users on Android 2.24.14.7 have spotted a new option to try out the Llama 3-405B model for Meta AI. While this option has currently been rolled out to beta users only and with significant limits on usage volume, it’s enough to get people excited about a full release, possibly in late July or August of 2024.
More from Tom's Guide
- I finally saw a live demo of ChatGPT-4o Voice — if anything it's underhyped
- Microsoft Build 2024: The biggest AI announcements and what they mean for you
- Gemini Live — what features are available now and what is coming soon
