Llama 3.2 Adds Vision and Smaller On-Device Models

At Meta Connect, Meta announced Llama 3.2 with new sizes and vision capabilities. Meta partnered with me on this video. I will review the announcement and show a brief test of the smallest model; I leave vision testing for a later video.

The 11B and 90B models add image understanding, while the 1B and 3B models are text-only and designed for more constrained devices. Meta presents the vision variants as compatible replacements for corresponding Llama 3.1 text models, subject to the actual deployment requirements.

I expect more AI computation to move onto phones, computers and other edge devices. Meta describes the smaller models as pretrained and instruction-tuned with a 128K context window, and lists optimization for Qualcomm and MediaTek processors.

Meta provides tooling for customization and local deployment, including torchtune and torchchat, along with access through its ecosystem. The Llama Stack is intended to simplify building applications across cloud, on-premises and device environments.

The stack’s announced interfaces cover inference, safety, memory, agent systems, evaluation and post-training among other functions. Meta lists several distribution and cloud partners, though availability and terms depend on the provider.

I review Meta’s benchmark tables comparing the 1B and 3B text models with small peers and the vision models with other models available at the time. The displayed scores are vendor-reported results on particular tests, not a guarantee of performance on every task.

Using Groq’s hosted 1B preview, I ask for a story and then a Python Snake game. I observe more than 2,000 tokens per second and report that the game works on the first attempt; this is a single hosted demonstration, not an on-device speed measurement.

Meta describes the vision models as supporting image reasoning, including charts, document understanding, captions and visual grounding. For example, a user might ask which month had the best sales based on a graph. I plan to test those capabilities separately.

The announced architecture connects a pretrained image encoder to the language model through adapter weights and cross-attention layers. Meta says it trained these on image-text pairs, updating the image encoder while keeping the language-model parameters fixed during adapter training.

For post-training, Meta describes supervised fine-tuning, rejection sampling and direct preference optimization, plus synthetic data generated with Llama 3.1. It also uses pruning and distillation to create smaller models, with the larger Llama model acting as a teacher.

I welcome the release and plan separate follow-up tests of text and vision intelligence. This video is an announcement overview and one informal text test, so broader claims about quality need further evaluation.