Architecture Designer — methodology

Contents

1. What it decides, and what it does not
2. The composition: which engine answers which question
3. Hardware Match’s answer is copied verbatim
4. The pipeline: stages, rates and the fan-out
5. Composing stages on one accelerator — the class E assumption
6. How the verdict is composed, and where it is capped
7. The rollups that are still legacy and class E
8. Legacy figures with no V2 owner, and what happened to each
9. What the result reports about its own coverage
10. What invalidates a result
11. Method changelog

1. What it decides, and what it does not

The Architecture Designer answers one question: given this workload, these constraints and this pipeline, what is the whole deployment — which platform, at how many nodes, with the cameras split how, running which model at each stage, against which constraints, with what bill of materials, and what should be measured before any of it is bought.

It does not decide which platform wins. That is Hardware Match's question, and its answer is copied here without change. It does not decide which model to run either — that is Model Match's, per stage. It does not compute a throughput, a latency, a memory figure, a thermal margin or a carrier verdict of its own: every one of those comes from the engine that owns it, stamped with that engine's name, so any number on the page can be reproduced by calling that engine directly with the same inputs. That is what makes the composed answer checkable rather than asserted.

What it does own is the composition itself: how the per-stage results are put together on one node, and what that composition is allowed to claim. There is exactly one judgement in that, it is unmeasured, and §5 is about nothing else.

2. The composition: which engine answers which question

Every constraint returned carries engine and check, its evidence class, its source records and the assumptions behind it. The result also carries sub_engines: the method version and dataset version of every engine that contributed, and a legacy flag on the four that are not V2 yet.

3. Hardware Match’s answer is copied verbatim

This is the rule the engine was built around, and it is worth stating in the flat form:

• No second ranking pass. There is no scoring function in this engine. The platform, the node count, the per-node camera assignment, the hardware cost and the hardware power are what Hardware Match returned; the five term scores and the weight table on the page are its own, published so the total can be recomputed from the printed numbers.
• No confidence re-pick. A LOW confidence on the winner is reported, never a reason to reach past it for the runner-up. The legacy orchestrator did exactly that — ranked, then spliced a multi-node economics winner to the front, then re-picked on a confidence tier — and the disagreements it produced are the reason this engine has one selection step instead of three.
• No mutation of the alternatives. alternatives[] is Hardware Match's ranked list minus the winner, in its order, carrying its own role labels — lower_cost, lower_power, higher_headroom, stretch. Nothing here re-sorts it, filters it, re-scores it or adds a role to it. The page renders it in the order it arrives.
• No override input. There is no field by which a caller picks the winner. Pinning a platform narrows the candidate set Hardware Match ranks and changes no other rule; it does not choose an answer, and if the pinned platform cannot hold the load the result says so instead of substituting one you did not ask about.

One consequence is worth being explicit about, because it looks like a bug until you see the alternative. Hardware Match ranks platforms against one model's load — the workload's model family and variant — because that is the question its five terms are defined over. The pipeline is then composed on the platform it picked. So a pipeline that does not fit on that platform is reported as not fitting, at whatever percentage of one accelerator it needs, rather than quietly triggering a second ranking pass against a different load. The result names the levers: reduce the fan-out, pin a smaller model, raise the node cap, or pin a platform.

4. The pipeline: stages, rates and the fan-out

A pipeline is up to four stages. Each stage carries a task, what it runs on, and a rate:

• A stage that runs on camera frames is sized at cameras on one node × the detect rate × the motion share. An explicit per-stage frames-per-second figure overrides that; left blank, the stage inherits the workload's detect rate.
• A stage that runs on an earlier stage's output is sized at the parent's rate × the fan-out, where the fan-out is how many detections each frame produces. It defaults to one per frame and the result marks it as defaulted when it was.

The fan-out is class E. Nothing measures how many objects your scenes contain; it is a statement about your deployment, not about the hardware, and it multiplies straight into the throughput the downstream stage is asked for — two detections per frame doubles that stage's required rate. It is therefore always shown, always labelled, and never silently chosen for you beyond the default of one.

Every stage is sized on one node, not on the fleet, because that is the machine the models actually share. A stage's model is Model Match's pick out of the models with a benchmark row on that platform, or Compute Fit's verdict on the model you pinned; either way, the throughput, latency, precision, runtime, batch and memory figures are that engine's own and are reproduced unchanged.

5. Composing stages on one accelerator — the class E assumption

When more than one stage lands on one node, they share one accelerator. This engine composes them by summing their utilizations and adding their memory on the shared memory budget. That is the whole model, and it has no measurement behind it.

There is no measured concurrent-model throughput anywhere in this corpus. That statement is precise, and it is worth unpacking, because two things that look like evidence for it are not:

• The multi-stream scaling curve models N streams of ONE model. It says what happens to throughput when you feed a single loaded network more concurrent streams. It says nothing about two different networks contending for the same compute units, the same memory bandwidth and the same scheduler — and treating it as if it did would be borrowing a number from a question it was never asked.
• The one multi-model source on record is a recommendation, not a measurement. It advises running the models in separate containers with CUDA stream priority rather than under MPS, and it budgets roughly 3 % container overhead. It does not measure the throughput either model achieves while the other is running. So it is cited as guidance and it is never turned into a coefficient; a 3 % container overhead is not a concurrency model.
• The stage vocabulary in the research corpus is decode → preprocess → infer → postprocess — the internal stages of running one model, not two models sharing a device.

Summing is the honest choice among the available ones. It is not conservative in every direction: real contention can be worse than a sum, because two resident networks also contend for memory bandwidth and cache, and it can be better, because a stage that is latency-bound rather than throughput-bound may interleave into the other's gaps. Which of those dominates on your silicon, your runtime and your models is exactly the thing nobody has measured here.

So the sum is stamped class E — engineering heuristic, no source record — on the pipeline block, on the pipeline_accelerator_share constraint and in the assumptions, and the result links to the measurement protocol, which is how you replace this assumption with a number. Memory is composed the same way and reports UNKNOWN rather than an estimate whenever a stage has no memory split on record — an unknown is never filled in with a guess.

6. How the verdict is composed, and where it is capped

The architecture's verdict is the worst of: Hardware Match's verdict on the winning platform, every stage's verdict, and the status of every composed constraint. A FAIL anywhere makes it NO FIT; a NEAR LIMIT makes it FIT WITH RISKS; an UNKNOWN caps it at NEEDS VALIDATION and never eliminates anything.

On top of that there is one hard cap. Whenever more than one stage shares an accelerator, the verdict is capped at NEEDS VALIDATION, regardless of how much headroom the sum shows. A class E sum can prove a miss — 155 % of one accelerator is a miss under any concurrency model — but it can never confirm a fit, because the arithmetic that would confirm it is the arithmetic nobody has checked. A single-stage architecture is not capped: nothing is composed, so there is no assumption to carry.

Each stage is also reported on its own, with its own verdict, so a reader can see which part of the composition is solid and which part is the assumption.

7. The rollups that are still legacy and class E

Four whole-system figures on this page do not come from a V2 engine and carry no source record. They are labelled class E, line by line, on the result and on the page:

The compute line is the exception: it is Hardware Match's, priced from a posted vendor listing, and carries its own class. The hardware cost and hardware power on the architecture block are likewise Hardware Match's, not the legacy rollup's — which is why those two figures and the whole-system rollup can differ on the same result and are shown separately rather than reconciled into one number.

A rollup that does not resolve is reported as null with a risk explaining why, never estimated. The legacy cost engine has no module row for several catalog platforms; on those the bill of materials is the compute line alone and the result says so.

8. Legacy figures with no V2 owner, and what happened to each

The engine this one replaces produced several figures that no V2 engine owns. Dropping them silently would hide a change in what the page claims, so each one is listed here with its disposition.

9. What the result reports about its own coverage

• Candidates, evaluated, feasible, excluded — and a per-registry count, so a platform that ten registries know well is distinguishable from one that three of them mention.
• Thin coverage is named. A platform present in three or fewer of the platform registries is still evaluated and still ranked, and is capped at NEEDS VALIDATION with the reason stated.
• Registry-only platforms are named. A platform that exists in a data registry but not in the hardware catalog is not a candidate, and the result lists which ones those are rather than leaving them silently absent.
• Stages resolved and rollups resolved are counted, so a partial answer is visibly partial.
• Every eliminated platform comes back with the exact constraint that eliminated it — the check, the engine, the status, what was required and what was available.

10. What invalidates a result

• A new benchmark row. Throughput here is the pessimistic edge of the rows that resolve for each model on this platform. One new measured row can move a stage's available fps, its utilization, its evidence class and therefore the platform ranking underneath the whole architecture.
• A firmware or JetPack change. A different JetPack, runtime or power mode changes what the configuration checker asserts, what throughput the benchmark rows apply to, and in some cases whether the configuration is supported at all. An architecture designed against one release is not evidence about another.
• A different fan-out. The detections-per-frame figure multiplies straight into every downstream stage's required throughput. If your scenes carry four objects where the result assumed one, the downstream stage needs four times the rate and the composition changes completely.
• A measurement that contradicts the sharing assumption. This is the one that matters most. The moment somebody measures two models running concurrently on this silicon, the summed utilization on this page is either confirmed or falsified — and if it is falsified, every multi-stage verdict computed from it moves. Until then the cap at NEEDS VALIDATION is the honest reading, and the measurement protocol is how you close it.
• Anything the sub-engines' own caveats cover. Software versions, thermal state, input resolution, batch size, the multi-stream efficiency curve, and the fact that every benchmark row is inference-only.
• The class E rollups. Network, storage, system power and the non-compute bill of materials are heuristics, not sourced figures. Do not put them in a purchase order without checking them against the vendor quotes.

11. Method changelog

Method changes bump the method version; the composite dataset version follows the newest of the registries the sub-engines read, and every result also carries the per-engine versions in sub_engines, with a legacy flag on the four rollup engines.