Home / Engines / Benchmark Reality Check

I measured X. The published number is Y. Why?

Enter what you measured for a hardware × model × precision and it is compared with the published expected range from the normalized benchmark database (or the estimator's fallback when nothing was measured). You get a verdict, the gap ratio, and the likely causes ranked by fit — each with the on-device command that settles it. Give the JetPack release and power mode and the Jetson Configuration Checker runs too.

01 · What you measured

Hardware & model
Hardware
Loading benchmark catalog…
Model family
Loading…
Precision
FP32
FP16
FP8
INT8
INT4
Your measurement
Metric
Frames per second
Latency (ms)
Tokens per second
Measured fps
Scope
Inference only
End-to-end pipeline
Advanced
Runtime
Batch
Resolution
Warm-up discarded
Not sure
Yes
No
Timed iterations
Your board (optional — sharpens the causes, runs the Jetson Configuration Checker)
JetPack / L4T
Select a Jetson hardware first
Power mode
Select a Jetson hardware first
Install method
Super flash configuration
Not sure
Yes
No
Cooling
Not sure
Active cooled
Passive
Enclosed
Clocks locked
Not sure
Yes
No
Deployment
Bare metal
Container
Select hardware, a model family and variant, and enter what you measured

03 · Submit a measurement

If your run followed the measurement protocol, submit it below — it is reviewed against the protocol before it appears as a class C row with your attribution. Run the check above first and use "Submit this measurement" to prefill this form.

Submit a measurement

04 · How this works

The reference is the Benchmark Explorer's expected range

For your hardware, model, variant and precision, the reference is the min / median / max of every published row (or the estimator's B → C → D fallback when nothing was measured). Your measurement is converted into the same metric if needed (fps ↔ latency at your batch size), then gap ratio = measured ÷ median. WITHIN_RANGE sits inside the range ±15%; BELOW_RANGE / ABOVE_RANGE fall outside it (direction-aware for latency, where lower is better); NO_REFERENCE means nothing exists to compare against yet.

Causes are ranked, not guessed

Each cause has a prior likelihood (from what you reported — power mode, cooling, clocks, scope, runtime, JetPack configuration) multiplied by how well its typical effect size matches your observed gap. Every cause carries the on-device command that settles it, its expected output, and a remediation; when you give a JetPack and power mode, the Jetson Configuration Checker's non-PASS checks feed directly into the ranking.

Submitting your measurement closes the gap for the next person

If your run followed the measurement protocol, submit it — it is reviewed before it appears as a class C row with your attribution, replacing an estimate or filling a hole in the database entirely.

Full formulas, source list and what invalidates a result: methodology. Reference rows: /benchmarks/. Measurement protocol: how to submit a comparable number. Query the database directly: Benchmark Explorer.

05 · FAQ

Why is my measured number lower than the published one?
Usually one of: a non-maximum power mode, clocks not locked, thermal throttling, an end-to-end pipeline measurement compared against an inference-only reference, a framework runtime instead of the optimized one, or — on Orin Nano/NX since JetPack 6.2 — a board not flashed with the Super configuration. Each cause below ranks by how well it fits your gap and names the exact command that confirms it.
Why is my measured number higher than the published one?
Common causes: the reference is an EdgeAIStack estimate rather than a measurement (class D or a fallback), newer JetPack/TensorRT than the reference row, a smaller input size or larger batch, or a lighter model variant / lower precision than the row you're comparing against.
What does NO_REFERENCE mean?
No published or derived row exists for this exact hardware, model, variant and precision — there is nothing to compare your number against yet. Submitting your measurement creates the first row.
What does the Jetson configuration check add?
When you give a JetPack release and power mode, the Jetson Configuration Checker runs against the same inputs and its non-PASS checks (unsupported pairing, version risk, needs validation) feed directly into the cause ranking — a documented configuration problem outranks a generic heuristic.