Gemma 4: Smaller Open-Weight Models for Local AI

Google has released Gemma 4, a new family of open-weight models. I welcome its continued investment in models that people can run and adapt locally. Google describes the family as designed for reasoning and agentic workflows, and I am excited to try it.

I think smaller, faster models can complement hosted frontier systems: use the most capable remote model for difficult tasks and local compute for many others. Hardware requirements depend on the model, precision, context and device; a parameter count alone does not guarantee that a particular setup will run it comfortably.

A displayed chart plots total model size against Arena Elo. Gemma 4 31B and 26B-A4B appear close to larger models on that particular preference measure. The 26B model is a mixture of experts with four billion active parameters; a similar plotted score does not mean the models are equivalent on every task.

I contrast the 31B model with much larger Qwen, GLM and Kimi models, some of which require substantial memory. I expect a smaller model to be more accessible for local use, although the precise hardware and quantization determine whether it fits and how quickly it runs.

The family has four sizes: effective 2B, effective 4B, a 26B mixture-of-experts model and a 31B dense model. I look up what “effective” means: the smaller models use per-layer embeddings, whose tables are used for lookups, so the active inference footprint is described separately from the total number of stored parameters.

Google says its 31B model ranked third among open models on the Arena AI text leaderboard at the time, with the 26B model sixth. I find the size-to-score relationship impressive. This is a snapshot of a specific leaderboard, not a universal ranking across tasks.

Google lists multi-step reasoning, function calling, structured JSON and system instructions as features for agent workflows. I want to test Gemma with OpenClaw. Google also presents it as an offline coding assistant, though I personally prefer hosted frontier models for demanding coding work.

According to Google, all models handle images and video, while E2B and E4B also accept audio for speech recognition and understanding. The edge models have a 128K context window and the larger models up to 256K. I am satisfied with the edge limit but had hoped for more context in the larger versions.

Google positions E2B and E4B for offline use on mobile and edge hardware, citing collaborations with Pixel, Qualcomm and MediaTek. It mentions devices such as phones, Raspberry Pi and NVIDIA Jetson Orin Nano. I speculate that such models could eventually appear on Apple devices; that is my prediction, not an announced deployment.

I point to a range of download and runtime options including Hugging Face, llama.cpp, MLX, Ollama and LM Studio, and encourage experimentation and fine-tuning. I also describe the release as commercially permissive and say Apache 2.0; that is my statement in the video, so users should check the actual model terms for their intended use.

I read several reported metrics for the 31B model, including an Arena text score of 1452, MMLU 85.2, AIME 2026 at 89%, LiveCodeBench at 80%, τ2-bench at 86% and GPQA Diamond at 84.3%. These are reported benchmark results, whose settings and scope matter when making comparisons.

I also show a ToolCall-15 test posted by Steve Vibe. The displayed results give 31B 30/30, 26B 27/30, E4B 22/30 and E2B 20/30. The 31B result is perfect on that specific test, not proof of flawless tool use in every real-world workflow.