vLLM/Recipes
Search
Previewof vllm-project/vllm#64586 at 90781b3.
NVIDIA

nvidia/Cosmos3-Nano

Compact 16B omnimodal world model (Mixture-of-Transformers) for multimodal understanding, world simulation, future prediction, action reasoning, and Physical AI

16B omnimodal world model — single-GPU H200 video/audio generation

View on HuggingFace
dense16B262,144 ctxvLLM 0.21.0+vLLM-Omninightlyomni
Default configuration: 8× H200 · BF16
vllm serve nvidia/Cosmos3-Nano \
  --host 0.0.0.0 \
  --port 8000 \
  --init-timeout 1800 \
  --omni

The interactive builder loads with JavaScript. Append .md to this page's URL, query string included, for any configuration's command as Markdown — e.g. /nvidia/Cosmos3-Nano.md.

Guide

Overview

Cosmos3-Nano is the compact (16B) member of NVIDIA's Cosmos3 family of omnimodal world models. Built on a Mixture-of-Transformers (MoT) architecture — an autoregressive tower for discrete-token generation paired with a diffusion tower for continuous modalities — it generates coherent text, images, video, audio, and action commands from combinations of text, image, video, and action-trajectory inputs. It targets Physical AI: world understanding, world simulation, future prediction, and action reasoning.

This recipe covers the vLLM-Omni serving path (video + muxed-audio generation). The same checkpoint also serves a text/multimodal Reasoner endpoint via standard vLLM — see Reasoner mode below.

Prerequisites

  • 1× H200 (or H100 / A100) for the documented single-GPU profile
  • The release-tested vllm/vllm-omni:cosmos3 container, or vLLM-Omni installed on top of final vllm==0.33.0 for this source checkout. The published Cosmos3 container and its historical results use an older runtime and do not qualify the current source. The separate Reasoner environment below, where present, retains its own plugin/runtime requirements.
  • Prompts should be JSON-upsampled for best quality (see cosmos-framework prompt upsampling)

Launch command (vLLM-Omni)

docker pull vllm/vllm-omni:cosmos3

vllm serve nvidia/Cosmos3-Nano \
  --omni \
  --host 0.0.0.0 \
  --port 8000 \
  --init-timeout 1800

To speed up inference with additional GPUs, enable context parallelism with --ulysses-degree or switch to tensor parallelism with --tensor-parallel-size. On memory-constrained GPUs, --enable-layerwise-offload reduces VRAM usage at a performance cost.

Example: text-to-video

curl -X POST http://localhost:8000/v1/videos/sync \
  -H "Accept: video/mp4" \
  -F "prompt=$(cat assets/example_t2v_prompt.json)" \
  -F "negative_prompt=$(cat assets/negative_prompt.json)" \
  -F "size=1280x720" \
  -F "num_frames=189" \
  -F "fps=24" \
  -F "num_inference_steps=35" \
  -F "guidance_scale=6.0" \
  -F "max_sequence_length=4096" \
  -F "flow_shift=10.0" \
  -F "seed=123" \
  --output output.mp4

Image-to-video uses the same endpoint with an input_reference file part. Audio is generated jointly and muxed into the output MP4 (48 kHz stereo AAC) — there is no separate audio endpoint.

Reasoner mode (multimodal understanding)

The same checkpoint serves a text/multimodal Reasoner for world understanding, future prediction, and action reasoning via the standard vllm package (not vLLM-Omni):

# CUDA 13 drivers:
uv pip install --torch-backend=cu130 "vllm==0.21.0" \
  "vllm-cosmos3 @ git+https://github.com/NVIDIA/cosmos-framework.git#subdirectory=packages/vllm-cosmos3" \
  openai
# CUDA 12.8 drivers: use --torch-backend=cu128 "vllm==0.19.1"

CUDA_VISIBLE_DEVICES=0 \
vllm serve nvidia/Cosmos3-Nano \
  --hf-overrides '{"architectures": ["Cosmos3ReasonerForConditionalGeneration"]}' \
  --tensor-parallel-size 1 \
  --mm-encoder-tp-mode data \
  --async-scheduling \
  --allowed-local-media-path / \
  --media-io-kwargs '{"video": {"num_frames": -1}}' \
  --port 8000

Then query the OpenAI-compatible /v1/chat/completions endpoint with text + image/video content.

References