Llama 3.2 1B Instruct: memory fit on every Jetson module.
Method v1.0 · dataset 2026-09-07 · verified 2026-09-07 · last updated September 2026
1.24 B parameters · 16 layers · 131,072-token max context. Hugging Face: meta-llama/Llama-3.2-1B-Instruct.
Model facts
| Fact | Value | Note |
|---|---|---|
| Parameters | 1.24 B | |
| Layers | 16 | |
| Hidden size | 2048 | |
| Attention heads | 32 | |
| KV heads | 8 | |
| Head dimension | 64 | |
| Max context | 131,072 tokens | |
| dtype | bfloat16 | |
| Vocabulary | 128,256 |
Hugging Face (mirror: unsloth/Llama-3.2-1B-Instruct) · hf:unsloth/Llama-3.2-1B-Instruct · config.json · verified 2026-09-07
“"num_hidden_layers": 16, "hidden_size": 2048, "head_dim": 64”
Parameters source
- Hugging Face (mirror: unsloth/Llama-3.2-1B-Instruct, identical weights to meta-llama repo) · hf:unsloth/Llama-3.2-1B-Instruct · verified 2026-09-07 · model.safetensors (blob metadata via HF API) · class A
“model.safetensors size 2471645608 bytes”
Published quantised artefacts
A file the publisher or a community mirror actually ships, used as the class-A weight figure when the requested quant matches. Any quant without a row here falls back to parameters × bytes-per-parameter (class D for block quants).
| Quant | Format | File | Size | Source |
|---|---|---|---|---|
| GGUF Q4_K_M / INT4 (AWQ, MLC q4f16) | GGUF Q4_K_M | Llama-3.2-1B-Instruct-Q4_K_M.gguf | 0.808 GB | bartowski (community GGUF) · hf:bartowski/Llama-3.2-1B-Instruct-GGUF · verified 2026-09-07 |
| INT8 / GGUF Q8_0 | GGUF Q8_0 | Llama-3.2-1B-Instruct-Q8_0.gguf | 1.321 GB | bartowski (community GGUF) · hf:bartowski/Llama-3.2-1B-Instruct-GGUF · verified 2026-09-07 |
Jetson tokens/s measurements
Published or archived throughput numbers, not modelled. Class C (external measured benchmark).
| Module | Runtime | Quant | Measured | Source |
|---|---|---|---|---|
| Jetson Orin Nano 8GB | MLC | INT4 (q4f16_ft) | 54.8 tok/s | NVIDIA Jetson AI Lab (archive) · jetson-ai-lab:tutorial_slm.html · verified 2026-09-07 |
| Jetson AGX Orin 64GB | MLC | INT4 (q4f16_ft) | 163.9 tok/s | NVIDIA Jetson AI Lab (archive) · jetson-ai-lab:tutorial_slm.html · verified 2026-09-07 |
Every module × every quantisation
Verdict and total memory at 4096-token context, 1 sequence, llama.cpp, headless. Q4 cells link to the static breakdown page; every other cell links to the live engine at that quantisation.
Notes
params_b computed as safetensors total_size / 2 bytes (bf16); meta-llama org repo returns 401 (gated) so unsloth mirror (byte-identical weights) used as source for config.json and file size.
Method and data: Model Memory Fit methodology. Full registry: model-memory.json.
Change the context, concurrency or runtime.
The live engine covers any context length, sequence count, KV precision and runtime, with a permanent link.