Qwen2.5-VL 3B Instruct: memory fit on every Jetson module.
Method v1.0 · dataset 2026-09-07 · verified 2026-09-07 · last updated September 2026
3.76 B parameters (text 3.13 B + vision tower) · 36 layers · 128,000-token max context. Hugging Face: Qwen/Qwen2.5-VL-3B-Instruct.
Model facts
| Fact | Value | Note |
|---|---|---|
| Parameters | 3.755 B | text 3.126 B + vision 0.629 B (estimated) |
| Layers | 36 | |
| Hidden size | 2048 | |
| Attention heads | 16 | |
| KV heads | 2 | |
| Head dimension | 128 | |
| Max context | 128,000 tokens | |
| dtype | bfloat16 | |
| Vocabulary | 151,936 |
Qwen (Alibaba) · hf:Qwen/Qwen2.5-VL-3B-Instruct · config.json · verified 2026-09-07
“"hidden_size": 2048, "num_attention_heads": 16, "num_key_value_heads": 2, "max_position_embeddings": 128000”
Parameters source
- Hugging Face · hf:Qwen/Qwen2.5-VL-3B-Instruct · verified 2026-09-07 · model.safetensors.index.json · class A
“"total_size": 7509245952”
Vision tower
| Fact | Value |
|---|---|
| Layers | 32 |
| Hidden size | 1280 |
| Patch size | 14 |
| Image size | — |
| Model type | — |
Vision config source
- Qwen (Alibaba) · hf:Qwen/Qwen2.5-VL-3B-Instruct · vision_config · verified 2026-09-07 · config.json vision_config · class A
“"depth": 32, "hidden_size": 1280, "num_heads": 16, "patch_size": 14”
mmproj (vision weights): GGUF mmproj Q8_0 (vision tower), 0.845 GB.
mmproj source
- ggml-org (llama.cpp maintainers) · hf:ggml-org/Qwen2.5-VL-3B-Instruct-GGUF · verified 2026-09-07 · Qwen2.5-VL-3B-Instruct-GGUF repo file listing · class A
“mmproj-Qwen2.5-VL-3B-Instruct-Q8_0.gguf size 844757728 bytes”
Image tokens: 1280 tokens per image (dynamic: (H/28)×(W/28) tokens after 2×2 spatial merge; 1280 assumed for a ~1 MP image; class D).
Published quantised artefacts
A file the publisher or a community mirror actually ships, used as the class-A weight figure when the requested quant matches. Any quant without a row here falls back to parameters × bytes-per-parameter (class D for block quants).
| Quant | Format | File | Size | Source |
|---|---|---|---|---|
| GGUF Q4_K_M / INT4 (AWQ, MLC q4f16) | GGUF Q4_K_M (text) | Qwen2.5-VL-3B-Instruct-Q4_K_M.gguf | 1.93 GB | ggml-org (llama.cpp maintainers) · hf:ggml-org/Qwen2.5-VL-3B-Instruct-GGUF · verified 2026-09-07 |
Jetson tokens/s measurements
Published or archived throughput numbers, not modelled. Class C (external measured benchmark).
No Jetson-measured tokens/s reported for this model yet.
Every module × every quantisation
Verdict and total memory at 4096-token context, 1 sequence, llama.cpp, headless. Q4 cells link to the static breakdown page; every other cell links to the live engine at that quantisation.
Notes
params_b (3.75B) is the FULL model (text backbone + vision encoder) from safetensors total_size; the '3B' name refers to the LLM backbone only. Vision GGUF requires a separate mmproj file loaded alongside the text GGUF in llama.cpp.
Method and data: Model Memory Fit methodology. Full registry: model-memory.json.
Change the context, concurrency or runtime.
The live engine covers any context length, sequence count, KV precision and runtime, with a permanent link.