Llama 2’s 2023 Release: Model Sizes, License and Safety

Meta has released Llama 2, and I see it as a major step for models with available weights. In July 2023 the release supports research and commercial use under Meta’s community license. It is not unrestricted open source: the license imposes conditions, and the comparisons in this discussion reflect models available at that time.

The release includes pretrained Llama 2 and dialogue-tuned Llama 2-Chat in 7 billion, 13 billion and 70 billion parameter sizes. Meta also trained a 34 billion parameter variant but did not release it at launch. The company reports training on Nvidia A100 GPUs, 40% more data than Llama 1 and a doubled context length of 4,096 tokens.

Meta uses grouped-query attention in larger models to improve inference scalability. The paper also discusses the electricity and estimated carbon emissions associated with training. I appreciate that the authors make the resource and environmental costs visible, even though any estimate depends on its accounting assumptions.

Microsoft was Meta’s preferred launch partner, even though Microsoft also invested heavily in OpenAI. Microsoft says it wants to give developers a choice of open and frontier models. I interpret the arrangement as a balance between supporting a broader model ecosystem and its relationship with OpenAI.

Llama 1 was released for research, whereas Llama 2’s community license permits commercial use subject to its terms. One provision requires a separate Meta license if a licensee’s or its affiliates’ products or services had more than 700 million monthly active users in the month preceding the July 2023 release. That is different from an automatic rule triggered simply when a new Llama-based product later reaches 700 million users.

I speculate that the large-user restriction protects Meta against its biggest competitors; that is my interpretation, not a stated finding about its intent. The license also has other conditions, so developers should read the actual terms. Although the weights are available, calling the release completely open source can obscure those restrictions.

From what I have seen at launch, Llama 2’s coding ability appears weaker than GPT-4’s. This is an early assessment, not a controlled comparison. Meta’s paper also acknowledges a performance gap with frontier models of the period, including GPT-4 and PaLM 2; the video’s examples do not establish that Llama 2 beats every ChatGPT model or task.

Safety is a major part of the paper. Meta reports red-teaming and a violation-rate evaluation, where lower is better under that test. The 34B chat model has a higher violation rate in the displayed comparison, and the paper says its release was delayed because there was insufficient time to red-team it adequately. This chart measures a particular evaluation, not complete safety in all uses.

I expect the community to produce fine-tuned variants with different behavior. On helpfulness and safety, Meta describes using separate reward models for the two objectives. I find the approach promising, but the video does not establish that it has permanently solved the tradeoff between helpfulness and safety.

At the time of this video, Meta made weights and code available through its distribution channels, and hosted 7B and 13B versions were appearing. I plan to test the different sizes and subsequent fine-tunes with my own rubric and report what I find. Availability and model rankings in this account are historical.