Reduce latency for RTSP streams

[kingdomsrl]

I'm building an user interface with multiple IP camera streams running at the same time. I'm using TRVCamMultiView with up to 16 viewers running. My problem is that the latency for all my cameras start from 1-2 seconds at program launch to 15-30 seconds after a couple of minutes.

This is my setup:
- Delphi 11.3 on Windows 11 x64
- FMX project, using FFMpeg
- Dahua PTZ IP camera, model DH-SD5A432XAN-HNR, supports H265, H264 and MJPEG over RTSP.

I'm using the substream 1 for these cameras for reduced bandwidth usage and this is how such stream is set up:
- Codec: H.264H (seems like an optimized codec)
- Resolution: 704*480
- FPS: 30
- Bitrate: 1024 Kb/S constant
- iFrame interval: 60

Things I have tried so far without improvement:
- Switching DeviceType between rvdtRTSP and rvdtIPCamera
- Switching VideoFormat between rvvfH264, rvvfAVI_H264 and rvvfMP4_H264
- FFMpegProperty.Audio:=False
- FFMpegProperty.TcpNodelay:=True
- FFMpegProperty.RecvBufferSize:=10000 (and smaller values)

Things I have tried but made the situation worse:
- FFMpegProperty.FrameDrop:=True (it causes the stream to end as soon as the first frame arrives)
- FFMpegProperty.RTSPTransport:=[rvpeUDP] (it causes the video to show artifacts, turns black and white, pixelated, jumping lots of frames)

I'm wondering why this latency "stabilizes" at 15-30 seconds and doesn't get larger while at the same time never recovering for real time video.

[Sergey Tkachenko]

1. Do you use the newest version of RVMedia?
2. Assign RVCamera.Latency = 0.

[kingdomsrl]

1. Do you use the newest version of RVMedia?

Yes, RVMedia FMX version 9.3

2. Assign RVCamera.Latency = 0

Done, video starts with almost no latency but in a couple of minutes it grows to 15-30 seconds again.

By the way the cameras have a propietary software (which I must replace) and there the latency seems minimal, like 1-3 seconds at most. When running both programs side-by-side video timestamps for propietary and RVMedia are the same, but after several minutes it grows to around 20 seconds (I have some screenshots but I don't know how to upload images here).

Something interesting I noted is that for this test I reduced the number of cameras from 16 to 6 and I took 10 minutes instead of 2 to reach that 20 sec latency. Will test with a single camera to see how it goes.

Edit: With a single camera latency between propietary and RVMedia is around 1 second and it stays the same.

[Sergey Tkachenko]

Theoretically, delay may be accumulated in the following places:
1. Inside the library that is used to get video (i.e., inside FFmpeg or GStreamer DLL)
2. In the code that works with this library and tries to keep a framerate of the original video.
3. Later, when decoded frames are buffered.

Delays at the step (3) must be eliminated when you assign RVCamera.Latency = 0.
Let's try to check the step (2). Open fmxMRVFFmpegObject.pas, and comment the lines:

Code: Select all

        if delay > 1 then
          Sleep(Round(delay));

Making delays between frames when decoding is important when displaying a prerecording video. But when displaying a stream from a camera, this delay is not necessary, because frames are received at real time.
Most probably, you use Windows 64-bit platform. In this case, your application is compiled with precompiled RVMedia units, and changes in source code can be applied only after recompiling packages. The simplest way to recompile them is running "Install RVMedia FMX in Delphi IDE" in Windows Start menu.

Delays at the step (1) may be reduced by assigning some FFMpegProperties. Or you can try using GStreamer instead of FFmpeg.

[kingdomsrl]

After commenting those pair of lines and recompiling packages (yes, I'm compiling for Win64) I can confirm latency is gone. RVMedia now is performing as good as the proprietary software, even when showing multiple streams simultaneously.
Thank you Sergey.

[cleytonmaiabr]

Send me the already modified file because I'm using a trial to test, it doesn't have the .pas to change, I'm facing the same delay problem... Thank you!!!

[cleytonmaiabr]

I'm using version 8.0

[Sergey Tkachenko]

In the newest versions, these delays can be removed without recompilation. Just assign RVCamera.FFMpegProperty.NoDelay = true.

[cleytonmaiabr]

Hello. I changed it but there is still a delay in the image, it would be something like 2 seconds, I have an old version of component 6, it does not have the delay, the camera movement is the same within the component, but the version has some image problems so I cannot use the version, which is fixed in version 8, but I have a problem with the delay. Any other suggestions?

[Sergey Tkachenko]

Do you have RVCamera.Latency = 0?

[cleytonmaiabr]

Yes... I changed it but nothing happened... the delay continues, I noticed something, when I uncheck the "UseFFMpegProperty" option it resets the delay, but the image is blurred. Any suggestions?

[cleytonmaiabr]

Hello... I downloaded version 10.3 trial from the website, declaring in FormCreate, RVCamera.FFMpegProperty.NoDelay = true. It works, when you mark it on the component nothing happens....

In version 8.0 that I have here, the name changes from NoDelay to TcpNodelay, but I declare it in the Form, and nothing happens, it continues with the Delay. Any suggestions?

[kingdomsrl]

In version 8.0 that I have here, the name changes from NoDelay to TcpNodelay

I think TcpNoDelay and NoDelay are dealing with distinct aspects of the system. TcpNoDelay is a network flag (available in 8.0) while NoDelay is a decoding flag (available from version 10.3). So version 10.3 is the way to go to as I don't think this fix will be backported to old versions.