Compute Fit — methodology

Contents

1. What it decides, and what it does not
2. The required load
3. The throughput resolver
4. Evidence class and demotion
5. The three latency tiers
6. What the database actually contains
7. The headroom target
8. Power modes
9. Precision, runtime and batch
10. The verdict ladder
11. The model ladder
12. What invalidates a result
13. Method changelog

1. What it decides, and what it does not

Compute Fit answers one question: on this platform, in this configuration, does the compute hold this load — and where does every number come from. It returns the required frames per second, the throughput the benchmark database resolves, the rows behind that figure with their sources, a latency reported by tier, and a ladder of what else would run on the same silicon.

It does not recommend a platform. The engines it replaces did: the legacy compute sizer answered a throughput question with a theoretical TOPS figure and then named a device, making it a third platform picker beside two others that already existed. Compute Fit takes one platform and answers for it, so its answer is checkable against that platform's own benchmark rows. When the question is which platform, that is Hardware Match.

It also does not model the rest of the pipeline. Every benchmark row in this database is inference-only: decode, preprocessing, tracking, encoding and the network are not in these numbers. Camera Stream Capacity models those stages, and an end-to-end latency budget is a separate question this engine deliberately does not answer.

2. The required load

Either a direct target throughput, or a camera load resolved by the sizing expression Frigate publishes for its own detectors: cameras × detect fps × the share of frames with motion. The default motion share is 1.0 — the worst case, every camera detecting at once — because a capacity plan that assumes quiet cameras fails on the day something happens in front of all of them.

The detect rate is not the camera's frame rate. A camera streaming at 30 fps whose detector runs at 5 fps needs five inferences per second from this engine and thirty decodes per second from another. Compute Fit sizes the first; the decode side belongs to Camera Stream Capacity.

3. The throughput resolver

There is one resolver and no second path. The Benchmark Explorer's expected range is asked for the hardware, model family, variant, precision and runtime; it reads the normalised database first and falls through to the estimator's benchmark hierarchy when a precision has no row, reporting the tier chain it tried. Nothing in Compute Fit reaches around it to a different table.

When rows exist, they are narrowed to the batch size asked for if any row was measured there — a batch-8 figure and a batch-1 figure are different measurements on the same silicon and mixing them into one range is meaningless. The lowest figure in the resulting set is the reported capacity, not the median and not the best row anyone published. Published benchmarks vary by batch, thermal state, software version and harness; the pessimistic edge is the number most likely to survive a real deployment. The full range is printed beside it so the disagreement is visible rather than hidden.

That reference figure is then multiplied by the multi-stream efficiency curve at the stream count (a class E scaling model) and by a power-mode calibration where one exists. Both factors are printed, so available = reference × efficiency × power factor can be recomputed from the numbers the result returns. A capacity figure whose arithmetic cannot be checked from what it publishes is not falsifiable.

4. Evidence class and demotion

The class is inherited from the rows: C when a named publisher measured and published the figure, D when it is EdgeAIStack-derived with the derivation stated. Classes A, B and E do not occur for a throughput in this corpus, and the engine never claims one.

A class C figure is demoted to D in two cases: when the rows were measured at a different input resolution than the one asked for, so the figure has been scaled across resolutions; and when a power-mode factor has been applied, because that factor is itself a class D calibration. Both demotions are stated in words on the check, and neither is silent.

5. The three latency tiers

This is the part the engine exists for. Of the benchmark rows that report a latency, all but sixteen report a number within two per cent of 1000 divided by the frame rate on the same row: the latency was computed from the throughput, not observed. The engine that Compute Fit replaces published that reciprocal as a latency, with no class, no source and no chain — at every tier, including the one where the throughput itself was a theoretical ceiling.

The fix starts at the data. Every row in the benchmark database is now stamped, at the point the database is built, with latency_source: measured when the figure was observed, derived_reciprocal when it equals 1000/fps to within two per cent. The p99 column carries latency_p99_source, and every p99 in the database is derived — a modelled 1.20–1.43× multiplier over the p50, never an observed tail. Compute Fit and its validation rules read that flag; neither re-derives the test at request time, and no page can present a reciprocal as a measurement.

The asymmetry in the two class D tiers is the point. A reciprocal is a lower bound on real latency: the real figure can only be worse. So a bound that already exceeds your target proves a genuine miss, while a bound under your target proves nothing at all — hence FAIL in the first case and REQUIRES VALIDATION, never PASS, in the second. The same rule is applied to the corpus-derived stated tier: a class D number cannot certify a target, only fail one.

Without a latency target no latency check runs at all, and the result reports a floor_ms explicitly labelled as the derived reciprocal. There is one further figure the engine carries but never uses as evidence: an internal Hailo p50/p99 table that exists in the code with no source record anywhere in the repository. It is reported as context, class D, with that caveat printed — it is not laundered into the check.

6. What the database actually contains

The honest limits, stated rather than filled. The database holds 567 rows over 21 hardware ids and 49 model families, of which 163 are class C and 404 class D. Only 166 of the 1,029 possible platform × family combinations carry a row at all, so a query outside that coverage lands on the estimator's fallback rather than on a measurement.

• Latency is thin everywhere. 539 of the 555 rows that report a latency computed it as 1000/fps. Sixteen were observed independently, and only three of those are class C rows with an external publisher behind them.
• Batch is thinner. 553 of 567 rows are batch 1. The only rows above it are fourteen Hailo rows at batch 8. Anything else is UNKNOWN, never extrapolated.
• Resolution is concentrated. 428 of 567 rows were measured at 640×640, so any other input size is a scaled figure and is reported class D.
• Power-mode joins are weak. Only a minority of Jetson rows resolve to a real nvpmodel preset, and the throughput calibration covers four platforms. A benchmark row is never joined to a preset the vendor does not ship.
• One catalog platform has no rows at all, and four hardware ids in the database are reference platforms outside the catalog. Both facts are reported rather than papered over.

None of this is filled by inventing numbers. It is filled by measurement: the submission protocol already accepts a latency figure, and a measured row on a real target replaces a modelled one the moment it lands.

7. The headroom target

The default is 20 %, and it is class E — engineering judgement, not measurement. It is chosen so the resulting utilization target, 0.80, is exactly the shared NEAR LIMIT threshold every other V2 engine already uses, rather than a second private threshold living in this engine alone.

It does one thing: it moves the NEAR LIMIT line on the throughput check. There is deliberately no separate headroom constraint, because on the default it would say exactly what the throughput check's own NEAR LIMIT status already says, and two thresholds that mean the same thing are how a stack ends up with four of them. Raise the target and a configuration that was a clean pass becomes NEAR LIMIT at the same utilization; nothing else in the result moves.

8. Power modes

The power-mode registry knows watts and clock ceilings but no throughput. The only mode-to-throughput relation in the repository is a calibration table covering four platforms, documented as part-measured and part-estimated. Compute Fit applies it only when both conditions hold: a calibration exists for that platform and mode, and the mode resolves to a published nvpmodel preset. Applying it demotes a class C throughput to D.

Otherwise — no calibration, or a mode that is not a stock preset on that module — the reference-mode figure is used unfactored and the result says so in a warning. It is never a validation error, and a benchmark row's own power-mode string is never joined to a preset without the registry agreeing that the preset exists.

9. Precision, runtime and batch

The precision check is a class E catalog assertion: the hardware catalog's supported-precision list carries an inline note rather than a source record, and the check says so. The runtime check is the same on non-Jetson platforms; on Jetson it is replaced by the Jetson Configuration Checker's sourced verdict, which cites NVIDIA's own release notes for the JetPack-to-runtime pairing.

A precision or runtime the platform cannot run returns UNSUPPORTED, makes the verdict NO FIT, and — importantly — resolves no throughput at all. The estimator hierarchy will happily scale a figure for FP16 on an INT8-only accelerator; a number about a configuration that cannot exist is a number about nothing, so the support checks gate the resolver rather than sitting beside it.

The batch check asks one thing: is there a row at this batch size. Batch 1 passes and says how many rows back it. A batch above 1 with a matching row passes and cites it. A batch above 1 without one is UNKNOWN — never a scaled batch-1 figure with a made-up factor.

10. The verdict ladder

The same ladder every V2 engine uses, applied to the checks above, in this order: any FAIL or UNSUPPORTED is NO FIT; any UNKNOWN, REQUIRES VALIDATION or VERSION RISK is NEEDS VALIDATION; any NEAR LIMIT is FIT WITH RISKS; otherwise FIT.

UNKNOWN never fails a configuration and never passes it free. It caps the verdict at NEEDS VALIDATION until somebody measures the thing that is unknown. In practice that means most results carrying a latency target land on NEEDS VALIDATION, because most latencies in the corpus are derived — which is an accurate description of the evidence, not a defect in the engine.

11. The model ladder

Instead of naming a different platform, the result answers what else would run here. Every model family, variant and precision with a benchmark row for that silicon is listed with the throughput it would give at the same efficiency and power factor, its evidence class, how many of its rows were measured, and whether it holds your load. The list is sorted by margin, and the combination you asked for is marked in place so you can see where it sits.

That is the useful half of what the legacy engine's platform recommendation was reaching for, answered with rows that exist for hardware you actually have.

12. What invalidates a result

• A latency that matters. If the deployment has a real latency requirement, treat every non-measured tier as a hypothesis and measure on the target — the reciprocal is a lower bound and the real figure includes queueing, preprocessing, copies and the rest of the pipeline that no row here contains.
• Software versions. A benchmark measured on one JetPack, TensorRT or SDK release is not portable to another; the rows carry their dates and, where the publisher stated one, their software version.
• Thermal state. Published benchmarks are usually short runs on a cool device. Sustained throughput on a passively cooled node in a warm enclosure can be materially lower — that is the Thermal Feasibility Checker's question.
• A modelled fallback. When no row matched, the figure came from a precision ratio, a cross-platform scaling or a theoretical ceiling. It is class D and it is a starting point for a measurement, not a capacity figure. A throughput modelled from a theoretical TOPS ceiling, or scaled onto a platform that has no benchmark row of its own, never reaches PASS: it can rule a configuration out (FAIL), otherwise the check is REQUIRES VALIDATION and the verdict NEEDS VALIDATION.
• Input resolution. Three-quarters of the corpus is 640×640; a figure scaled to another input size assumes the scaling model holds for that model and that silicon.
• The multi-stream efficiency curve and the headroom target are both engineering judgement, and a workload whose concurrency pattern differs from the modelled one will differ from this answer.
• Everything outside inference. Decode, preprocessing, tracking, encoding, storage and network are not in these numbers at all.

13. Method changelog

Method changes bump the method version; the composite dataset version follows the newest of the registries this engine reads, and every result also carries the per-engine versions in sub_engines.