Research deep-dive / Video infrastructure

RTSP stream optimization for video AI workloads

RTSP stream optimization means choosing the right stream, codec, resolution, frame rate and transport for each analytics task rather than accepting camera defaults. On a retrofit site it typically cuts decode load and bandwidth by an order of magnitude, because most detection work needs far less video than recording does.

Reviewed by Dr. Vivek Gujar, Chief Strategy Officer  ·  31 August 2026  ·  14 min read

Most analytics deployments on existing camera estates are sized wrongly, and almost always in the same direction. An appliance is specified against the number of cameras, connected to the recorder, pointed at whatever stream the recorder offers first, and then found to be undersized. The diagnosis is usually recorded as insufficient compute. The actual cause is that the system was asked to decode far more video than the task required.

This deep-dive works through the five decisions that determine how much video a system actually has to handle: which stream, what resolution, which codec, how many frames per second, and which transport. Each is examined against published standards and peer-reviewed measurement rather than vendor guidance, because vendor defaults are set for recording and human review, not for machine consumption.

What RTSP controls, and what it does not

The first source of confusion is the protocol itself. RTSP is a control protocol. It negotiates a session and carries commands such as DESCRIBE, SETUP, PLAY, PAUSE and TEARDOWN. The media travels separately, normally over RTP, defined in RFC 3550. RTSP 2.0, specified in RFC 7826 and published in December 2016, formally obsoletes the original RFC 2326 from 1998.

RFC 7826 is explicit that the protocol deliberately avoids assuming anything about the media it controls. The specification states that RTSP exists to provide access to multimedia content but tries to be agnostic about the media type and the delivery protocol used. That agnosticism is why the same protocol serves a doorbell camera and a broadcast contribution feed, and also why the interesting engineering decisions all sit outside RTSP itself.

The practical consequence is that a session can negotiate perfectly over RTSP and still deliver no usable video, because the failure is in the media path. It also means that RTSP itself is almost never the bottleneck. What matters is what you asked the camera to encode.

Decision one: which stream

Nearly every IP camera published in the last decade encodes the same sensor output two or three times simultaneously and publishes each encoding at a different RTSP path. A typical configuration offers a main stream at 1920x1080 and 25 fps, and a sub stream at 640x360 and around 10 fps.

The recorder consumes the main stream because recording is an evidentiary function. Analytics has no such requirement, and the difference in cost is substantial.

Decision two: how much resolution the task needs

The instinct that more resolution is safer produces systems that cost several times what they need to. The disciplined alternative is to specify against the operational task, which is exactly what IEC 62676-4, the application guidelines part of the video surveillance standard, exists to do.

The standard anchors named operational tasks to a minimum pixel density on the target plane, expressed in pixels per metre. The classic four-step DORI ladder sets detection at 25 px/m, observation at roughly 63 px/m, recognition at 125 px/m and identification at 250 px/m. The model derives from the number of pixels needed across a human face, taking an average face width of 16 cm.

The 2025 revision of IEC 62676-4 adds a finer seven-step ladder alongside the original four, running from Overview at 20 px/m through Outline, Discern, Perceive, Characterise and Validate to Scrutinise at 1500 px/m. The four DORI thresholds are unchanged, so existing designs remain valid.

Two conclusions follow, and they point in opposite directions.

The first is that a great deal of analytics work sits at the bottom of this ladder. If the task is to know that a person entered a zone, the standard puts the requirement at 20 to 25 px/m. A 640x360 sub stream covering a 10 metre wide scene delivers 64 px/m, comfortably above detection and into observation. Running that task on a main stream is spending twenty times the compute for no gain in outcome.

The second is that plate reading and face work sit at the very top and cannot be served by a sub stream at all. The 2025 Scrutinise tier at 1500 px/m is an order of magnitude above what a general surveillance camera delivers at typical standoff distances. This is why ANPR normally requires a dedicated, tightly framed camera per lane rather than a share of an existing wide view.

Decision three: codec, and why the fifty percent headline misleads at 1080p

H.265, also called HEVC, is widely described as delivering the same quality at half the bitrate of H.264. That claim has a solid provenance. The JCT-VC verification testing programme in 2013 measured an average bitrate saving of around 50 percent at matched Mean Opinion Score. A BBC research team examining ultra high definition content found an average saving of 59 percent when judged subjectively, against 44 percent when judged by objective PSNR-based metrics.

Both figures are real, and both are misleading when applied to video surveillance, for a reason that is easy to miss: the saving depends heavily on resolution, and it is smallest at the resolutions surveillance actually uses.

A 2024 comparison of VVC, AV1, HEVC and AVC across high resolutions found that the bitrate savings between HEVC and H.264 start at around 14 percent at full HD and rise to roughly 77 percent at 8K.

There is a second consideration specific to analytics. H.265 decode is more computationally expensive than H.264. On an edge appliance without hardware HEVC decode, moving the estate to H.265 saves network bandwidth and spends compute, which on a device already constrained by decode is the wrong trade. Verify hardware decode support for the specific codec and profile before switching an estate over.

Decision four: frame rate

Frame rate is the cheapest optimisation available and the most frequently overlooked. Analytics load scales close to linearly with frames analysed, and most tasks do not need every frame.

Sampling guidance by task type, assuming typical indoor and gate scenarios:

Dropping an estate from 25 fps to 5 fps for detection tasks reduces analytic load by a factor of five, on top of any saving from stream selection. Combined with the 22 to 1 pixel throughput ratio from sub stream selection, the compound effect on a detection-only workload is large enough to change the hardware class required.

Decision five: transport

RTP can travel over UDP or be interleaved within the RTSP TCP connection. The trade is conventionally described as latency against reliability, and the conventional description is incomplete.

RFC 7826 is notably cautious about interleaving. The specification advises that interleaving should generally be avoided unless necessary, because it complicates operation and adds overhead, and it warns that head-of-line blocking may cause problems.

For a human watching a monitor, that guidance is sound. For analytics, the calculus shifts, because the two failure modes have very different consequences. A dropped frame under UDP is a missed detection opportunity. A delayed frame under TCP is a detection that arrives late. In most analytics tasks other than real-time control, late is recoverable and missing is not.

There is also a finding from the measurement literature that runs against intuition. A 2023 study in Sensors compiled a dataset of 11,200 full HD and ultra HD sequences encoded in H.264 and H.265 at five bitrates, with simulated packet loss from 0 to 1 percent, and assessed them both objectively and subjectively. Alongside the expected result that quality falls as loss rises, the authors found that the quality of sequences affected by packet loss declines with increasing bitrate.

This matters for analytics because a corrupted frame does not merely look bad, it produces detections that are wrong rather than absent, which is considerably harder to notice in an alert log than a gap.

The constraint that surprises people: concurrent connections

Every optimisation above assumes the appliance can open the streams it wants. Cameras impose a limit on simultaneous RTSP sessions, commonly between four and twenty depending on model and firmware, and the NVR already occupies one. Older cameras sit at the low end of that range.

NVR re-streaming is frequently more constrained than the cameras behind it, sometimes capped well below the channel count. Establish the limit during survey, not during commissioning, because exceeding it produces intermittent failures that present as network problems and are diagnosed slowly.

Security, and the gap between the specification and the field

RTSP 2.0 tightened security materially over RTSP 1.0. RFC 7826 mandates TLS support, requires both Basic and Digest authorisation to be supported, and requires that agents handling RTP media implement Secure RTP as defined in RFC 3711.

Almost no deployed surveillance estate reflects this. The overwhelming majority of installed cameras speak RTSP 1.0 with credentials embedded in the URL and media in the clear. An independent assessment of the protocol family observes that basic RTSP is suitable only for local managed networks, having no packet loss recovery, dynamically negotiated UDP ports and poor firewall behaviour.

The operational conclusion is not to replace the estate. It is to place it on an isolated VLAN with no inbound internet route, to reach it from analytics inside that segment rather than through a port forward, and to issue per-consumer credentials so a decommissioned system can be revoked without rotating the camera password and rediscovering everything that used it.

Putting it together

The general principle is that continuous work should be cheap and expensive work should be rare. Detection runs constantly and therefore runs on the cheapest stream that satisfies the pixel density requirement for the task. Evidence capture runs occasionally and can afford the main stream. Designing the two paths separately, rather than running everything at recording quality, is what makes analytics on an existing estate economically sensible.

For the architectural argument behind processing on site at all, see our note on edge AI. For the protocol mechanics in brief, see RTSP, and for discovery across a mixed estate, ONVIF.

Frequently asked questions

References

1. Schulzrinne, H., Rao, A., Lanphier, R., Westerlund, M., and Stiemerling, M. (Eds.), Real-Time Streaming Protocol Version 2.0, RFC 7826, IETF, December 2016.
2. Schulzrinne, H., Casner, S., Frederick, R., and Jacobson, V., RTP: A Transport Protocol for Real-Time Applications, RFC 3550, IETF, July 2003.
3. IEC 62676-4, Video surveillance systems for use in security applications, Part 4: Application guidelines. Pixel density thresholds summarised in Axis Communications, Pixel density based on IEC 62676-4:2014.
4. Bienik, J., Uhrina, M., Sevcik, L., and Holesova, A., Impact of Packet Loss Rate on Quality of Compressed High Resolution Videos, Sensors, 2023. DOI 10.3390/s23052744.
5. Catellier, A., et al., Characterization of the HEVC Coding Efficiency Advance Using 20 Scenes, NTIA Institute for Telecommunication Sciences, IEEE ISM 2015.
6. Performance Comparison of VVC, AV1, HEVC, and AVC for High Resolutions, Electronics, 13(5), 953, MDPI, 2024.
7. BBC R&D subjective testing of HEVC against AVC for UHD content, as reported in 2015.