The short version
Jetson-class deployments succeed or fail on four things outside the model: power and thermal design, a zero-copy camera pipeline, a reproducible JetPack and TensorRT build, and secure fleet updates with rollback.
Jetson Orin Nano makes on-device inference cheap enough for ordinary industrial and retail deployments. NVIDIA's developer kit documentation covers bring-up and JetPack setup well, and TensorRT covers optimisation. What no datasheet covers is the fleet you have to keep alive afterwards — that is where hardware engineering earns its money.
Power and thermals decide your real throughput
Orin Nano modules run at selectable power modes, and sustained throughput depends on the thermal solution around them, not the benchmark number. In a sealed enclosure on a factory wall at 45 °C ambient, an under-designed heatsink turns into silent clock throttling — the model still runs, frames just quietly drop. Validate at maximum ambient, in the real enclosure, with the real duty cycle, and log clock and thermal state as production telemetry so degradation is visible.
The camera pipeline, not the model, is the usual bottleneck
- —Keep frames on the GPU: capture, convert, infer and encode without round-tripping to CPU memory.
- —Decide MIPI CSI versus GigE/USB early — it changes the carrier board, cable lengths and EMC behaviour.
- —Pin exposure, gain and white balance for inference; auto modes make accuracy drift with the weather.
- —Time-sync cameras when multiple views must be fused, and stamp frames at capture, not at inference.
- —Budget frames, not FLOPS: at ten cameras and fifteen frames per second, a five-millisecond copy per frame is your headroom gone.
Model optimisation with a quality gate
| Step | Typical effect | Risk to check |
|---|---|---|
| FP16 conversion | Large speed-up, minimal accuracy change | Numeric range in custom layers |
| INT8 quantisation | Further speed and memory gain | Accuracy loss on rare classes — calibrate on representative data |
| Layer fusion / TensorRT engine build | Latency reduction | Engines are version- and device-specific; rebuild in CI |
| Input resolution reduction | Cheapest win available | Small-object detection collapses first |
Run the accuracy suite on the optimised engine on the target device, never on the workstation build. A TensorRT engine is tied to the JetPack and hardware combination it was built for, so treat engine builds as artefacts produced by CI against a pinned JetPack version.
Fleet operations from day one
- 01Immutable, signed images with A/B partitions and automatic rollback on failed boot or failed health check.
- 02Per-device identity and attestation, so a stolen device cannot join the fleet.
- 03Staged rollouts: one device, then a canary group, then the fleet.
- 04Telemetry that matters at the edge: temperature, clocks, frame drops, inference latency percentiles, model version, disk wear.
- 05An offline mode that is the normal mode — buffer locally, reconcile when a network appears.
Choose the production module before the enclosure is tooled. Developer kits and production modules differ in connectors, power and thermal interface, and discovering that after mechanical sign-off costs a full board revision.
What we would do in a pilot
One instrumented device on your real footage and real enclosure: optimised engine with an accuracy report on-device, thermal validation at maximum ambient, camera pipeline benchmarks, and a signed OTA path with rollback.
Sources
- 01Jetson Orin Nano Developer Kit User Guide — NVIDIA
- 02JetPack SDK setup — NVIDIA
- 03TensorRT documentation — NVIDIA
Figures drawn from our own engagements elsewhere on this site are illustrative placeholders. Anything attributed above links to its public source.