<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Seamless video-to-video transition with v4l2h264dec + waylandsink on Pi3 (Debian Trixie) — frozen frame between clips]]></title><description><![CDATA[<h5>Platform: Raspberry Pi 3B, Debian Trixie, Qt 5.15, GStreamer 1.24, Wayland compositor, v4l2h264dec</h5>
<h4>Why we built a custom GStreamer item instead of using Qt Multimedia's Video element</h4>
<p dir="auto">Before describing the transition problem itself, I want to explain why we're not using Qt Multimedia's Video QML element, since that's the obvious first question.</p>
<p dir="auto">Qt Multimedia's GStreamer backend does use v4l2h264dec — we confirmed this with GST_DEBUG=3. That's not the issue. The issue is what happens at EOS on this specific platform.</p>
<p dir="auto">On Pi3 / kernel 6.6, when GST_EVENT_EOS reaches the V4L2 sink, it triggers a known memory leak in the Pi's V4L2 driver at end-of-playback. For a digital signage player looping content 24/7, this CMA (Contiguous Memory Allocator) leak accumulates until V4L2 buffer allocation fails:</p>
<pre><code>v4l2h264dec: Failed to allocate output buffers — Cannot allocate memory
</code></pre>
<p dir="auto">The workaround is to intercept and drop GST_EVENT_EOS via a pad probe on the sink pad, before it reaches the driver. End-of-video is detected via GST_MESSAGE_EOS on the bus (which is still emitted normally).</p>
<p dir="auto">Qt Multimedia's Video element doesn't expose any hook to install a pad probe at this level. This is why we built <strong>GstVideoItemV2</strong>, a custom QQuickItem that drives the pipeline directly with the raw GStreamer C API.</p>
<p dir="auto">Our pipeline:</p>
<pre><code>filesrc → qtdemux → h264parse → v4l2h264dec → queue → waylandsink
                                                             ↑
                                               pad probe: drop GST_EVENT_EOS
</code></pre>
<h4>Note on Qt6:</h4>
<p dir="auto">migrating to Qt6 is not on our roadmap today, but if Qt6 Multimedia's GStreamer backend exposes pad-probe-level hooks, or if newer kernels have fixed this V4L2 EOS leak, it would be a strong argument to prioritize the upgrade. Any feedback on Qt6 + Pi3 + Wayland experience is welcome.<br />
The problem: ~13–80ms frozen frame at every video transition<br />
In a playlist zone, our player pre-builds the next video's pipeline while the current video is still playing (buildPipeline() runs on a separate dedicated worker thread). When video 1 ends:</p>
<pre><code>Video 1:  PLAYING ──────────────────────── STATE_NULL (~9ms, holds v4l2 lock)
Video 2:     buildPipeline (11ms, no lock) ──────── waits lock ── set_state(PLAYING)
Screen:   video 1 last frame ─────────────────────── FROZEN ─────── video 2 starts
</code></pre>
<p dir="auto">The freeze is not a black screen — it's video 1's last frame held by the Wayland compositor while video 2's waylandsink hasn't rendered its first frame yet.</p>
<p dir="auto">Loop works fine: when the same video loops (seek to 0), we reuse the same pipeline and the same waylandsink — no teardown, no lock, no freeze. The problem is strictly video1 → video2.</p>
<p dir="auto">Why we can't keep waylandsink alive between videos<br />
Reusing waylandsink across different source files (to avoid teardown) causes additional CMA exhaustion after ~10–15 cycles on this kernel. The V4L2 decoder's DMA buffers are tied to the sink's Wayland buffer pool and aren't released until STATE_NULL. We must destroy and recreate the full pipeline on every transition.</p>
<p dir="auto">One hardware decoder, one at a time<br />
The Pi3 has one V4L2 H.264 decoder instance (/dev/video10). We serialize access with a static QMutex that gates both the STATE_NULL release (video 1's worker thread) and the set_state(PLAYING) acquisition (video 2's worker thread).</p>
<p dir="auto">Current mitigation: worker thread pre-warming<br />
Each GstVideoItemV2 runs all GStreamer ops on a dedicated QThread worker. On Pi3 ARMv7 @ 1.4 GHz, pthread_create → kernel stack allocation → first event loop dispatch takes ~40–50ms. We pre-create the next video's QQuickItem (and its worker thread) during playback of the current video. Best-case gap: ~13ms. Still perceptible.</p>
<h3><strong>Questions</strong></h3>
<p dir="auto">Atomic waylandsink handover: is there a GStreamer pattern to swap the source decoded into a single persistent sink without going through <strong>STATE_NULL —</strong> on a system where the sink can't be kept alive across sources due to CMA constraints?<br />
input-selector: has anyone used it upstream of a shared sink with two independent decoder chains on Pi3 / kernel 6.6, managing the CMA budget?<br />
kmssink + DRM atomic flip: viable on Pi3 + Trixie to bypass compositor buffering and hide the handover gap?<br />
STATE_PAUSED vs STATE_NULL: does PAUSED release V4L2 CMA buffers on this kernel? If it holds the hardware open (preventing dual-open) but releases CMA, we could keep a warm pipeline.<br />
<em><strong>Qt6 Multimedia:</strong></em> does it handle the V4L2 EOS CMA leak differently on Pi3/Wayland? Any first-hand experience?<br />
Thanks in advance for any pointers.</p>
]]></description><link>https://forum.qt.io/topic/165090/seamless-video-to-video-transition-with-v4l2h264dec-waylandsink-on-pi3-debian-trixie-frozen-frame-between-clips</link><generator>RSS for Node</generator><lastBuildDate>Wed, 30 Sep 2026 01:10:35 GMT</lastBuildDate><atom:link href="https://forum.qt.io/topic/165090.rss" rel="self" type="application/rss+xml"/><pubDate>Mon, 14 Sep 2026 09:16:48 GMT</pubDate><ttl>60</ttl><item><title><![CDATA[Reply to Seamless video-to-video transition with v4l2h264dec + waylandsink on Pi3 (Debian Trixie) — frozen frame between clips on Mon, 14 Sep 2026 19:04:20 GMT]]></title><description><![CDATA[<p dir="auto">Hi,</p>
<p dir="auto">I can only answer on the last part: Qt6 Multimedia module currently uses ffmpeg as backend on all platform except for embedded Linux which might be your case. I suggest you take a look at <a href="https://doc.qt.io/qt-6/qtmultimedia-gstreamer.html" target="_blank" rel="noopener noreferrer nofollow ugc">GStreamer chapter in the Qt Multimedia documentation</a>. There might be stuff in there that could be of interest.</p>
<p dir="auto">Hope it helps</p>
]]></description><link>https://forum.qt.io/post/840148</link><guid isPermaLink="true">https://forum.qt.io/post/840148</guid><dc:creator><![CDATA[SGaist]]></dc:creator><pubDate>Mon, 14 Sep 2026 19:04:20 GMT</pubDate></item></channel></rss>