Meta's Third-Generation Llama3 LLM

Published on April 22, 2024

Meta has recently announced the release of its next-generation large language model (LLM), Llama3. This state-of-the-art open-source LLM is designed for developers, researchers, and businesses to build, experiment, and responsibly scale their generative AI ideas.

Llama3 is the latest iteration of Meta's open-access Llama family. It is available in two sizes: 8 billion (8B) and 70 billion (70B) parameters. These models are pretrained and instruction-fine-tuned, supporting a broad range of use cases. According to standard benchmarks, Llama3's 8B and 70B parameter models are the most powerful models of their size available in the open for free. Meta claims that Llama3 performs better than Claude Sonnet, Mistral Medium, and GPT-3.5.

Llama3 introduces several new features and capabilities. It demonstrates state-of-the-art performance on a wide range of industry benchmarks and offers improved reasoning. In addition, it introduces new trust and safety tools with Llama Guard 2, Code Shield, and CyberSec Eval 2.

A significant change in Llama3 compared to Llama2 is the use of a new tokenizer that expands the vocabulary size to 128,256 (from 32K tokens in the previous version). This larger vocabulary can encode text more efficiently and potentially yield stronger multilingualism. In support of Meta's longstanding open approach, Llama3 is being put in the hands of the community. Meta aims to kickstart the next wave of innovation in AI across the stack-from applications to developer tools to evaluations to inference optimizations and more.

Meta is dedicated to developing Llama3 in a responsible way. In the near future, they plan to make Llama3 multilingual and multimodal, have longer context, and continue to improve overall performance across core LLM capabilities such as reasoning and coding. With the release of Llama3, Meta continues to push the boundaries of what is possible in the field of AI. As an open-source tool, Llama3 is set to drive innovation and development in the global AI community. It will be exciting to see the new applications and advancements that will emerge from this powerful tool.

Sources:

https://ai.meta.com/blog/meta-llama-3/

https://mspoweruser.com/meta-releases-llama-3-the-most-powerful-open-llm-model/

https://llama.meta.com/

https://huggingface.co/blog/llama3