Gemma 4

Gemma 4 is Google DeepMind’s most capable open-weight model family to date, released in 2025. Built from the same world-class research and technology underpinning Gemini 3, Gemma 4 was designed to deliver frontier-class intelligence at a fraction of the parameter cost — making it viable for local deployment, edge devices, and cost-sensitive production environments alike.

The family is distributed under an Apache 2.0 license, meaning it can be used, modified, and redistributed freely, including for commercial purposes.

Model Sizes and Variants

Gemma 4 ships in four configurations, each targeting a different deployment profile:

• E2B (Effective 2B): The smallest variant, optimized for on-device and edge inference. Supports native audio input alongside image and text.
• E4B (Effective 4B): A step up in capability while remaining highly portable. Also supports audio, image, and text inputs with a 128K context window.
• 26B MoE (Mixture of Experts): A sparse model with 26 billion total parameters but only 4 billion activated per token during generation. This delivers the compute efficiency of a ~4B model during inference while retaining the knowledge capacity of a much larger one. All 26B parameters must still reside in memory to enable fast routing.
• 31B Dense: The flagship model. Ranked #3 on the Arena AI text leaderboard among all open models globally at launch. Supports a 256K context window.

Architecture

Gemma 4’s architecture is built on the Transformer foundation with several notable design choices:

• Alternating attention layers: The models interleave local sliding-window attention (efficient for nearby token relationships) with global full-context attention (for long-range dependencies), balancing compute cost with contextual breadth.
• Mixture of Experts (26B variant): Uses sparse activation — only a subset of “expert” sub-networks fire per token — dramatically reducing FLOPs per forward pass without sacrificing model capacity.
• Extended context windows: 128K tokens for the edge models (E2B, E4B) and up to 256K tokens for the 26B and 31B variants, enabling document-level and multi-document reasoning.

Multimodal Capabilities

All Gemma 4 models are natively multimodal:

• Vision: Every model in the family can process images at variable resolutions, enabling tasks like image captioning, visual question answering, diagram interpretation, and document understanding.
• Video: Native video understanding is supported across the family, allowing temporal reasoning over sequences of frames.
• Audio (E2B, E4B): The two smaller models include native audio input, supporting speech recognition, spoken language understanding, and audio-conditioned generation.
• Text: The models are pretrained on data in over 140 languages, making them one of the most multilingual open model families available.

Reasoning and Agentic Use

Gemma 4 was explicitly designed with agentic workflows in mind:

• Multi-step reasoning: The models demonstrate strong performance on math, logic, and multi-hop reasoning benchmarks — core requirements for autonomous agents that plan and execute sequences of actions.
• Function calling: Native support for structured tool/function invocation, enabling integration with external APIs, databases, and services without prompt engineering workarounds.
• Structured output: Built-in support for generating valid JSON, making it straightforward to wire Gemma 4 into pipelines that expect machine-readable responses.
• System instructions: Native handling of system-level prompts allows developers to set persistent behavior, personas, and constraints without occupying user-turn context.

Performance Benchmarks

• The 31B Dense model ranks #3 among all open models on the Arena AI text leaderboard.
• The 26B MoE model ranks #6 on the same leaderboard.
• The models show marked improvements over prior Gemma generations on instruction-following, mathematical reasoning, and code generation benchmarks.

Deployment and Availability

Gemma 4 is available through multiple channels:

• Google AI for Developers (ai.google.dev) — API access and model cards
• Google Cloud — Vertex AI and Cloud Run deployments
• Hugging Face — All model weights hosted publicly under Apache 2.0
• LM Studio — Local desktop inference
• Ollama, llama.cpp, and other local runtimes — Compatible with standard quantization formats (GGUF, etc.)

Open-Weight vs. Open-Source

Gemma 4 is released as open-weight rather than fully open-source — the model weights are freely downloadable and usable under Apache 2.0, but the training data, data processing pipelines, and full training code are not publicly disclosed. This is an important distinction for teams evaluating provenance, reproducibility, or compliance requirements.

Relation to Gemini

Gemma 4 is architecturally derived from Gemini 3 but is not the same model. Gemini models remain proprietary and are accessible only through Google’s APIs, while Gemma 4 models are fully downloadable and self-hostable. The relationship is analogous to how Meta’s LLaMA models relate to their internal research models — shared lineage, different distribution and licensing model.

Use Cases

Given its combination of multimodality, long context, agentic features, and open weights, Gemma 4 is well-suited for:

• On-device AI assistants — particularly E2B and E4B on mobile or embedded hardware
• Document intelligence — ingesting and reasoning over long PDFs, contracts, or reports
• Code generation and review — strong instruction-following and structured output support agentic coding workflows
• Multilingual applications — 140+ language support out of the box
• Custom fine-tuning — open weights allow domain adaptation without API dependency
• RAG pipelines — large context windows reduce the pressure on retrieval precision