Phi-4-mini Instruct: memory fit on every Jetson module.
Method v1.0 · dataset 2026-09-07 · verified 2026-09-07 · last updated September 2026
3.8 B parameters · 32 layers · 131,072-token max context. Hugging Face: microsoft/Phi-4-mini-instruct.
Model facts
| Fact | Value | Note |
|---|---|---|
| Parameters | 3.8 B | |
| Layers | 32 | |
| Hidden size | 3072 | |
| Attention heads | 24 | |
| KV heads | 8 | |
| Head dimension | 128 | |
| Max context | 131,072 tokens | |
| dtype | bfloat16 | |
| Vocabulary | 200,064 |
Microsoft · hf:microsoft/Phi-4-mini-instruct · config.json · verified 2026-09-07
“"num_attention_heads": 24, "num_key_value_heads": 8, "partial_rotary_factor": 0.75”
Parameters source
- Microsoft · hf:microsoft/Phi-4-mini-instruct · verified 2026-09-07 · Phi-4-mini-instruct README.md · class A
“the 3.8B parameters Phi-4-mini-instruct model was compared with a set of models”
Published quantised artefacts
A file the publisher or a community mirror actually ships, used as the class-A weight figure when the requested quant matches. Any quant without a row here falls back to parameters × bytes-per-parameter (class D for block quants).
| Quant | Format | File | Size | Source |
|---|---|---|---|---|
| GGUF Q4_K_M / INT4 (AWQ, MLC q4f16) | GGUF Q4_K_M | Phi-4-mini-instruct-Q4_K_M.gguf | 2.492 GB | unsloth (community GGUF) · hf:unsloth/Phi-4-mini-instruct-GGUF · verified 2026-09-07 |
Jetson tokens/s measurements
Published or archived throughput numbers, not modelled. Class C (external measured benchmark).
No Jetson-measured tokens/s reported for this model yet.
Every module × every quantisation
Verdict and total memory at 4096-token context, 1 sequence, llama.cpp, headless. Q4 cells link to the static breakdown page; every other cell links to the live engine at that quantisation.
Notes
head_dim left null: hidden_size/num_attention_heads=3072/24=128 nominal, but config sets partial_rotary_factor=0.75 (rotary applied to 96 of 128 dims) -- could not find an explicit head_dim field to quote, so not stated as a hard number. bartowski/Phi-4-mini-instruct-GGUF returned 401 (repo not found under that name); unsloth/Phi-4-mini-instruct-GGUF used instead.
Method and data: Model Memory Fit methodology. Full registry: model-memory.json.
Change the context, concurrency or runtime.
The live engine covers any context length, sequence count, KV precision and runtime, with a permanent link.