Published answers to this question range from 4 streams to 100, which tells you the question is underspecified rather than hard. Capacity is bound by throughput, not memory, and it is bound in two independent places — the decode engines and the inference compute. You are limited by whichever runs out first, and they run out at different points. The biggest lever is the one almost nobody pulls: how many frames per second you actually analyse, which for most tasks is far fewer than the camera produces.
It is the first question on every video analytics project and it is usually answered with a number, confidently, by someone who has not asked what resolution you run. The honest answer is that the number is a property of your deployment rather than of the hardware.
Here is the spread from publicly available figures, all for roughly the same question. One source reports an RTX 3070 with a single decode engine saturating around 12 to 14 H.264 1080p30 streams. Another estimates about 34 decode streams per GPU at 1080p, but only 22 concurrently analysed at ten analysed frames per second. A third suggests an L4 handling 60 to 100 with a small INT8 model and proper batching. A Jetson AGX Orin is put at 8 to 16. And one independent benchmark found that processing every frame of every camera in real time held only to about 4.
Those are not contradictions. They are different hardware, different models, and — most importantly — different definitions of what it means to handle a stream.
Two ceilings, not one
The reason sizing goes wrong is that teams reason about it as a single capacity. It is two, and they are enforced by different parts of the chip.
The decode ceiling
Video decoding runs on dedicated hardware engines rather than the general compute cores, which means it has its own independent capacity. A reasonable estimate is total decode capacity divided by per-stream decode cost, where that cost rises with resolution, frame rate and codec.
Because it is separate silicon, you can saturate decode while the compute cores are barely working. That situation is common, mildly maddening, and the reason a monitoring dashboard showing 30% GPU utilisation can coexist with dropped frames. NVIDIA publishes decode session counts by codec and resolution for each GPU generation, and that documentation is the place to start rather than any blog — including this one.
The inference ceiling
The compute side is driven by model size, numeric precision, batch size and how many frames you actually run through it. Two things matter disproportionately here.
Batching across cameras. The advice is unusually consistent: group frames from multiple cameras into one inference call rather than running a separate inference instance per camera. One source estimates the per-camera approach is around four times less efficient, and the reason is structural — a GPU is a throughput device and a batch of one wastes most of its parallelism. The cost is a little latency while frames wait to be grouped, which matters for a safety interlock and is irrelevant for a dwell-time count.
The cap is not one stage. One benchmark found throughput levelling off around 220 to 235 frames per second from eight streams onward, and noted that the ceiling was not attributable to inference alone — preprocessing, inference and tracking all grow with batch size. Sizing that models only the detection step will overestimate capacity.
The lever nobody pulls
Here is where most deployments leave capacity on the table.
There is a large gap between the frames a camera produces and the frames your use case needs. The figures above make it visible: one source describes roughly 34 decodable streams dropping to 22 analysed at ten analysed frames per second — already a third of the source rate. Most detection tasks need far less than even that.
A PPE compliance check does not need thirty samples a second. A person does not enter and leave a zone in 33 milliseconds, and the rule layer that turns detections into alerts is usually requiring persistence over seconds anyway — which we went through in PPE detection on a factory floor. Dropping from 30 analysed frames per second to five multiplies the cameras a GPU can serve, and for most use cases costs nothing measurable in detection quality.
Before buying a second GPU, it is worth asking what frame rate the use case genuinely requires. The answer is frequently "far less than we are running," and that conversation is free.
How to size it properly
Measure, on your own conditions. Every number in circulation — including every number in this article — assumes a specific resolution, codec, model and definition of processing a stream, and none of those will match yours exactly.
Take a representative sample of your real feeds at their real resolution and codec. Run your actual model at the analysed frame rate your use case needs. Increase the stream count until something saturates, and note which ceiling gave way first, because that determines what you buy more of — a decode-bound deployment and a compute-bound one want different hardware.
Then leave headroom. A site that adds cameras is the normal case, not the exception, and a system sized exactly to today's camera count becomes a procurement conversation the first time someone mounts another one.
The same principle applies here as to accuracy figures: a number measured on someone else's setup describes their setup. We made that argument about detection quality in why vision accuracy drops between the demo and the floor, and it holds just as firmly for throughput.
Questions we get asked
How many streams can one GPU handle?
Published figures differ by more than an order of magnitude: ~12–14 on an RTX 3070 decode-bound, ~34 decode but 22 analysed at 10 fps elsewhere, 60–100 on an L4 with a small INT8 model and batching, 8–16 on Jetson AGX Orin, and ~4 for full real-time on every frame. Different hardware, models, and definitions of "handle".
Memory or throughput?
Throughput, nearly always. Detection models are small relative to modern GPU memory. What runs out is processing capacity, in two separate places — the decode engines and the inference compute — and which binds depends on resolution, codec, model and analysed frame rate.
Why is decode a separate ceiling?
Because decoding runs on dedicated hardware engines, not the compute cores. Estimate it as total decode capacity over per-stream cost, which rises with resolution, frame rate and codec. You can be decode-bound with compute idle — which looks like an underused GPU. NVIDIA publishes decode session counts per generation; start there.
Does batching matter?
Substantially. Group frames from multiple cameras into one inference call rather than one instance per camera — one source puts the per-camera approach at roughly 4× less efficient, since a batch of one wastes the GPU's parallelism. The cost is a little latency, which matters for an interlock and not for a dwell count.
What's the biggest lever?
Analysed frame rate, and almost nobody pulls it. A PPE check doesn't need 30 samples a second — nobody crosses a zone in 33ms, and the rule layer requires persistence over seconds anyway. Going from 30 analysed fps to 5 multiplies camera count at no measurable cost in quality.
How should we size it?
Measure on your own feeds, at your resolution and codec, with your model, at the frame rate the use case needs. Raise stream count until something saturates and note which ceiling gave way — that decides what to buy more of. Then leave headroom, because sites add cameras.
We size and build on-premise video analytics against real camera feeds rather than spec sheets. If someone has quoted you a cameras-per-GPU number without asking your resolution, codec and analysed frame rate, it was not a calculation.