Meta unveils Llama 3 generative AI models

Meta has unveiled the latest iterations of its Llama series of open generative AI models: Llama 3.

The new models, Llama 3 8B and Llama 3 70B, is expected to be a significant leap forward in performance compared to their predecessors, Llama 2 8B and Llama 2 70B.

The new models are trained on custom-built 24,000 GPU clusters, these models boast 8 billion and 70 billion parameters, respectively, making them among the most potent generative AI models available today.

Meta stated that Llama 3 8B and Llama 3 70B exhibit outstanding performance across various AI benchmarks, including MMLU, ARC, and DROP, which evaluate the models’ knowledge, skill acquisition, and reasoning abilities.

Meta claims that Llama 3 8B outperforms other open models like Mistral 7B and Gemma 7B on several benchmarks, while Llama 3 70B competes favorably with flagship generative AI models like Gemini 1.5 Pro.

In addition to performance enhancements, Meta highlights the use of a larger training dataset for Llama 3 models, comprising 15 trillion tokens. However, the sources of this data remain undisclosed.

Meta said it has implemented new data-filtering pipelines and updated safety suites like Llama Guard and CybersecEval to address concerns regarding toxicity and bias.

Despite these advancements, restrictions apply to the use of Llama models.

Developers are barred from using Llama models to train other generative models, and special licenses are required for app developers with more than 700 million monthly users.

Looking ahead, Meta announced it’s plans to further enhance the capabilities of the Llama 3 series, striving for multilingual and multimodal capabilities, longer context understanding, and overall performance improvements.

While the Llama 3 models are available for research and commercial applications, they are not entirely open source.