Llama 3.1: Meta’s 405B Model and the Open-Weight Ecosystem

Meta has announced Llama 3.1, including a 405-billion-parameter model and updates to its 70B and 8B models. I am excited that a model with downloadable weights appears competitive with leading closed models on some evaluations. That is a July 2024 assessment, not proof that it wins every task or will remain at the frontier.

I describe the release as open source, but “open weight” is more precise: Meta provides downloadable weights under a license with conditions, and it did not release all training data. The Open Source Initiative has argued the Llama 3 family’s license is not open source. See [its assessment](https://opensource.org/blog/metas-llama-license-is-still-not-open-source).

I am especially interested in the smaller 8B model, which can be more practical for local and edge use than 405B. Meta says the new models support a 128K context window and eight languages. A larger window allows more input, but does not guarantee perfect retrieval or reasoning over the entire context.

One use for a large model is creating synthetic examples to train or distill a smaller one. I compare this possibility with NVIDIA’s earlier Nemotron work: a team can use generated data to adapt a model without obtaining every example from a human-labeled dataset. The resulting quality still depends on the data, the process and independent evaluation.

Meta is presenting Llama as an ecosystem rather than just a model release. It describes a reference system for building agents, Llama Guard 3 and Prompt Guard safety tools, and a request for comment on a Llama Stack API. I hope the interfaces will make it easy to swap providers and integrate with existing tools.

Meta lists more than 25 partners, including cloud and hardware companies, and says users in the United States can try 405B through Meta AI and WhatsApp. I plan to test it more thoroughly later. Availability and partner offerings described here reflect the 2024 announcement.

I show a table circulating as a leaked benchmark comparison between Llama 3.1, previous Llama versions and GPT-4o. The post itself says the leak may be real, and the video does not verify its origin or testing methodology. It suggests strengths and weaknesses on selected tasks, but cannot establish that any model is universally better.

The same table appears to show substantial gains for the 8B model. Berman says HellaSwag rose from 46 to 67, but the visible table shows approximately 46.2% to 76.8%; it shows HumanEval from 34.1% to 68.3%. Those are values in the unverified table, not necessarily Meta’s official measurements. Meta’s [model card](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/MODEL_CARD.md) reports results under its own evaluation settings.

I imagine using a local 8B model for routine prompts and routing harder requests to a larger hosted model. I estimate it might handle 95–98% of prompts, but that is my speculation rather than a measured rate. Local inference can work offline once models and software are installed, although hardware, power and setup still have costs.

I also cite venture capitalist Bill Gurley’s criticism of AI regulation and his view that established closed-model companies seek rules that protect their position. That is his political argument and does not establish the motives of any company or legislator. I return to the model release after noting the debate.

Meta says the upgraded 8B and 70B models support longer context, multilingual use and tool use. It also says the license permits developers to use Llama outputs to improve other models. This could make synthetic-data and distillation workflows more accessible, subject to the actual license terms.

Meta reports evaluating models on more than 150 benchmark datasets and describes 405B as competitive with leading models across a range of tasks. It says 405B was trained on over 15 trillion tokens using more than 16,000 NVIDIA H100 GPUs. These are Meta’s reported training and evaluation details, not independent proof of performance in all uses.

The 405B model is large enough that local deployment requires substantial memory and compute. Meta says it quantized the model from 16-bit to 8-bit to lower inference requirements and run it on a single server node. A server node is not necessarily a consumer laptop, and further quantization or hardware configurations involve their own tradeoffs.

For post-training, Meta describes supervised fine-tuning, rejection sampling and direct preference optimization, using synthetic data for much of the supervised fine-tuning data. I wonder whether 405B outputs were used to improve the 8B model, but the clip does not establish that specific training relationship.

I ask the hosted model to write Snake in Python, copy its output and run it. The game starts and responds during my short test, including restarting after a collision. That is a useful example of code generation, not a systematic assessment of coding reliability or security.

Meta’s Llama Stack proposal aims to standardize interfaces for components such as fine-tuning, synthetic data, inference and agent applications. I think releasing models and tools can also encourage developers to build in Meta’s ecosystem, giving the company a role in shaping standards. That is my interpretation of its business incentives.

Downloadable weights give developers ways to customize and deploy models themselves, within the license and available resources. Meta also describes inference, evaluation, retrieval, function calling and other ecosystem components. I welcome the release and plan a more complete comparison of 405B and 8B, while acknowledging that the brief demonstration here cannot establish universal frontier performance.