NVIDIA Cosmos 3 Edge

NVIDIA Cosmos 3 Edge is a 4-billion-parameter open world model, the smallest of three scales in NVIDIA’s Cosmos 3 family, purpose-built to run vision reasoning and robot action generation directly on edge hardware. Announced July 20, 2026 alongside NVIDIA’s SIGGRAPH releases, Cosmos 3 Edge is the on-device counterpart to the much larger Cosmos3-Nano (16B) and Cosmos3-Super (64B) variants: instead of shipping camera frames to a datacenter and waiting for a response, a robot running Cosmos 3 Edge reasons about what it sees and produces its next batch of control actions locally, on a Jetson Thor or comparable module, at real-time control rates. That shift matters for physical AI generally: latency to a remote GPU is a liability the instant a robot’s actuators depend on the answer, and Cosmos 3 Edge is NVIDIA’s answer to “make the world model small and fast enough to live where the robot lives.”

Why an Edge-Scale World Model

A world model, in the Cosmos sense, is a system trained to predict how a scene will evolve, given an action, well enough to be useful for planning and control, not just to render a plausible-looking video. The datacenter-scale members of the Cosmos family (including Cosmos3-Super at 64B parameters) push this toward maximum fidelity: better long-horizon rollout accuracy, richer scene understanding, more robust generalization across environments. That quality has a cost: 64B parameters do not fit, and would not run at useful frame rates, on the compute budget available inside a mobile robot or an autonomous vehicle’s onboard compute tray.

Cosmos 3 Edge exists to close that gap without simply shrinking the datacenter model until it degrades. Built on a dense 2-billion-parameter transformer core and post-trained into a 4B-parameter world action model, it targets robot-control resolution (640x360 observations) rather than cinematic output resolution, trading visual fidelity the downstream controller does not need for the inference speed it does need.

Architecture: Two Towers, One Shared Attention Stack

Cosmos 3 Edge uses a dual-transformer design NVIDIA describes as “two transformer towers” that share multimodal attention layers while keeping separate normalization and MLP components:

graph TD
    classDef default fill:#ffffff,stroke:#4338CA,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef data    fill:#EEF0F7,stroke:#0D9488,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef process fill:#F7F8FC,stroke:#6366F1,stroke-width:2px,color:#0F172A,rx:8px,ry:8px;
    classDef output  fill:#4338CA,stroke:#4338CA,stroke-width:2px,color:#ffffff,rx:8px,ry:8px;

    IN([Camera frame + text/action tokens]):::data --> SHARED[Shared Multimodal Attention Layers]:::process
    SHARED --> AR[Autoregressive Tower: vision/text understanding]:::process
    SHARED --> DIFF[Diffusion Tower: vision, audio, action generation]:::process
    AR --> REASON([Reasoning tokens]):::output
    DIFF --> STEPS{Denoising steps}
    STEPS -->|distilled: 4 steps| FAST[Fast path, ~25x speedup]:::process
    STEPS -->|base: 50 steps| SLOW[Base path, higher fidelity]:::process
    FAST --> ACT([32 action vectors per inference]):::output
    SLOW --> ACT

The autoregressive tower handles understanding: parsing the incoming vision and text tokens and producing reasoning about the scene. The diffusion tower handles generation: it processes vision, audio, and action tokens together to synthesize the next visual prediction or the next batch of robot actions. Sharing the multimodal attention layers between the two towers lets the model reuse a single learned representation of “what’s in this scene” for both understanding and generation, rather than training two disconnected sub-models that would each need their own scene representation learned from scratch.

Actions themselves are represented as compact geometric vectors encoding translation, rotation, and manipulation (e.g., gripper) state, giving a direct, structured link between a predicted visual change and the physical control input that would produce it, rather than requiring a separate policy network to reinterpret a video prediction into motor commands.

Step Distillation: 50 Steps Down to 4

Diffusion-based generation ordinarily requires dozens of denoising steps to go from noise to a clean output, which is fine for offline video generation but incompatible with real-time robot control. NVIDIA released step-distilled versions of Cosmos 3 Edge that cut inference from 50 denoising steps down to 4, reported as up to 25x faster than the base model. This is the mechanism that makes on-device, real-time deployment viable at all: without distillation, a 50-step diffusion pass per control cycle would blow through any robot’s control-loop latency budget long before the fourth or fifth denoising step even finished.

Interactive: Denoising Steps vs. Inference Speed

Step count is a real trade-off, not a free lunch: fewer denoising steps means faster inference but a rougher approximation of the full diffusion trajectory, and different tasks tolerate that approximation differently. Drag the slider to see how the reported ~25x speedup scales as you move between the 50-step base model and the 4-step distilled checkpoint.

Hardware and Real-Time Performance

Cosmos 3 Edge is supported across NVIDIA’s RTX PRO, DGX, GeForce RTX, and Jetson lines, including the Jetson Thor and Jetson T2000/T3000 edge modules. On Jetson Thor, the distilled model achieves real-time control at 15 Hz, generating 32 actions per inference call, enough to drive a continuous control loop for tasks like pick-and-place manipulation without stalling on a stale action batch.

# Scenario: streaming control loop on Jetson Thor at ~15 Hz.
import time

while robot.is_running():
    frame = camera.read_frame()
    result = pipe(image=frame, prompt=current_task_prompt, num_inference_steps=4, num_actions=32)
    for action in result.actions:
        robot_controller.execute(action)
        time.sleep(1 / 15)  # pace to the model's real-time control rate

Benchmarks

Cosmos 3 Edge ranks #1 on VANTAGE-Bench among 4B-parameter vision analytics models, and NVIDIA reports state-of-the-art results for robot policy learning at this parameter scale. A separately distilled image-to-video variant built on the same Cosmos 3 family ranked #1 on the Artificial Analysis Image-to-Video leaderboard as of July 2026, evidence that the distillation approach used for Edge does not simply trade away quality for speed across the board.

Use Cases

• Robot manipulation policies: Cosmos3-Edge-Policy-DROID is a released checkpoint post-trained specifically for pick-and-place tasks using the DROID robot manipulation dataset.
• Smart infrastructure monitoring: on-site vision reasoning without streaming raw camera feeds to a datacenter.
• Manufacturing and warehouse automation: real-time action generation for arms and mobile robots operating on local compute.
• Surgical robotics simulation: evaluating candidate actions against a learned world model before they reach a physical instrument.
• Autonomous vehicle onboard reasoning: scene understanding and short-horizon action prediction within the vehicle’s own compute budget.

Getting Started

Weights are available on Hugging Face as nvidia/Cosmos3-Edge and the DROID-specialized nvidia/Cosmos3-Edge-Policy-DROID. Cosmos 3 is fully supported in the Hugging Face diffusers library via the Cosmos3OmniPipeline class, and NVIDIA’s own Cosmos GitHub repository and companion cosmos-framework repository provide inference cookbooks (cookbooks/cosmos3/run_inference.py) and post-training paths for teams adapting the base model to a specific robot or task.

What’s New (2025-2026)

• World models are being explicitly tiered for deployment target, not just scaled up. The Edge/Nano/Super split treats “runs on the robot” and “runs in the datacenter” as separate design points from the start, rather than releasing one large model and hoping quantization gets it small enough later.
• Aggressive step distillation is becoming standard for real-time diffusion deployment. The 50-to-4-step, ~25x speedup pattern used here mirrors a broader 2025-2026 trend of distilling diffusion models specifically for latency-critical settings (robotics, live video) rather than treating step count as fixed.
• Physical AI is consolidating around shared world-model backbones. Rather than training a separate model per task (perception, planning, control), Cosmos 3 Edge folds vision reasoning and action generation into one architecture, following the same direction as physical AI efforts elsewhere in the industry.
• Edge deployment of foundation-model-scale reasoning is now a shipping product, not a research demo. Real-time 15 Hz control on Jetson Thor from a model trained with datacenter-scale techniques marks a concrete point where “small enough to run on the robot” and “smart enough to be useful” have converged.

Practical Guidance

# Scenario: a warehouse pick-and-place arm needs the next 32 control
# actions predicted from its current camera frame, running entirely
# on an on-prem Jetson Thor box (no cloud round-trip).

# 1. Pull weights and clone the inference cookbook:
#    huggingface-cli download nvidia/Cosmos3-Edge
#    git clone https://github.com/NVIDIA/cosmos-framework

from diffusers import Cosmos3OmniPipeline
import torch

pipe = Cosmos3OmniPipeline.from_pretrained(
    "nvidia/Cosmos3-Edge",
    torch_dtype=torch.bfloat16,
).to("cuda")

# Distilled checkpoint: 4 denoising steps instead of 50, ~25x faster
result = pipe(
    image="dock_camera_frame.png",
    prompt="pick up the small box and place it on the conveyor",
    num_inference_steps=4,
    num_actions=32,          # actions generated per inference call
)

# Each action is a compact geometric vector: translation, rotation,
# and gripper/manipulation state, ready to feed a robot controller.
for action in result.actions:
    robot_controller.execute(action)