Meta just announced Llama 3.1
Meta has just announced the release of Llama 3.1, which comes in three versions: a large model with 405 billion parameters, a middle version with 70 billion parameters, and a small version with 8 billion parameters. The large model, with its 405 billion parameters, is now the largest open-source language model (LLM) to date, rivaling leading models such as GPT-4, GPT-4o, Gemini 1.5 Pro, and Claude 3.5 Sonnet.
LLMs are increasingly specializing in specific domains. For instance, GPT-4o, "o" standing for "omni", is model capable of handling text, speech, and video, while Sonnet is celebrated for its unique feature called Artifacts.
The most significant updates in Llama 3.1 include improved non-English language support compared to its predecessor, Llama 3, and an extended context window of 128k tokens, on par with GPT-4o. Additionally, Llama 3.1 boasts better quality and quantity of training data. An interesting aspect of this model is its ability to use generated synthetic data in the training process. The potential benefits of synthetic data are a hot topic among researchers.
Benchmark tests indicate that Llama 3.1 405B is one of the best LLMs available, competing closely with models like Claude 3.5 Sonnet and GPT-4o.
Source: Meta