QMediaPlayer jittery in PyQt6. FFmpeg "surface pool size" warnings.

[donquibeats]

Hello.

I'm in the process of updating an application previously running with PyQt5.14 so that it runs with PyQt6.6.

I've gone with 6.6 instead of 6.7 for the moment because I saw mentions online that 6.7 doesn't support h264 video, and from a very quick test I did that appeared to be true.

The code is mostly unchanged, with QMediaPlaylist having been removed, but essentially it's still the same QMediaPlayer code as before, playing the same video files as before. However, the playback is much much worse.

The playback sticks whenever the process is busy, which is a big no-no unfortunately for this application which needs frame-accurate playback (at 24/25fps with 720p video, so not too taxing). The video and audio often go out of sync, with the video lagging behind the audio. There are often blocky decompression artefacts on the video panel too, a bit like what you would see with digital TV transmitted across a poor analogue reception. Absolutely none of these problems happened with Qt5, and when I swap back to Qt5, everything is absolutely fine again.

In the console output, FFmpeg churns out warning messages like this:

[[AVHWFramesContext @ 000001D46BFBBA00] Static surface pool size exceeded.
[h264 @ 000001D46B9791C0] get_buffer() failed
[h264 @ 000001D46B9791C0] thread_get_buffer() failed
[h264 @ 000001D46B9791C0] decode_slice_header error
[h264 @ 000001D46B9791C0] no frame!]

So my question is unfortunately a big vague, as it's such a big problem at the moment, but... does anyone recognize these symptoms? Could anyone suggest why the QMediaPlayer in 6.6 is performing so poorly compared to Qt5 under the same conditions? Are there any settings or config items (for example with this "static surface pool size") that I can access and tweak to see if it helps?

Any help would be very much appreciated, as at the moment, I'm finding QMediaPlayer unusable in Qt6.6.

[JoeCFD]

@donquibeats Qt5.14 uses gstreamer
By default, Qt6 uses FFmpeg. And Qt Multimedia module was rewritten in Qt6. Therefore, you can not expect the same performance.

[donquibeats]

Thanks. I was aware of the FFmpeg change, but I'm disappointed to see that, in my use case, it's had a big negative effect on performance.

When you say "by default" Qt6 uses FFmpeg, are there other options? Is gstreamer still an option available in Qt6?

[JoeCFD]

@donquibeats Maybe you can try to change the backend to gstreamer with
export QT_MULTIMEDIA_BACKEND=gstreamer

Qt 5 Multimedia module does not work well either. Better to write your own player with raw gstreamer or ffmpeg code directly for better controls.

[donquibeats]

Thanks, I'll give that a go.

I will Google it of course, but I don't suppose anyone has already shared code examples for the raw gstreamer or ffmpeg approaches?

[JoeCFD]

@donquibeats
Qt 5 example:
https://github.com/GStreamer/gst-plugins-good/tree/master/tests/examples/qt
Qt 6 is similar because only qml sink is changed.

[donquibeats]

Fantastic, thank you. I'm going to take time to get my head around that example and see what I can come out with