Gstreamer overlay set window handle QT6 not working

[sdickerson]

Hi!

I'm having an issue where even though I use gst_video_overlay_set_window_handle() to set the window of a gstreamer pipeline to a qt video widget, it's still making its own window. However, not for all pipelines.

gst_init(nullptr, nullptr);

    // [2] Prepare the pipeline
    pipeline = gst_parse_launch(
        "playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm",
        NULL);  // works for this pipeline, output is in desired window
    //pipeline = gst_parse_launch(
    //    "videotestsrc pattern=snow ! video/x-raw,width=1280,height=720 ! autovideosink",
    //    NULL); // does not work for this pipeline, gstreamer creates its own window

    // [4] Set video overlay
    gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(pipeline), vw->winId());

    // [5] Run the pipeline
    GstStateChangeReturn sret = gst_element_set_state(pipeline, GST_STATE_PLAYING);
    if (sret == GST_STATE_CHANGE_FAILURE)
    {
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        // Exit
        QTimer::singleShot(0, QApplication::activeWindow(), SLOT(quit()));
    }

Any help with this issue would be greatly appreciated!! :)

[SGaist]

Hi,

What if you remove the sink from your second pipeline ?
You did not set one on the first.

Out of curiosity, why not use the QML sink plugin from GStreamer if you don't use the Qt Multimedia module ?

[sdickerson]

Hi!
My bad, i forgot to mention thats a function. I do use Qt Multimedia, the class declaration is as follows

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>
#include <QMediaPlayer>
#include <QtMultimediaWidgets/qvideowidget.h>
#include <QFileDialog>
#include <QProgressBar>
#include <QSlider>
#include <QComboBox>
#include <QLayout>
#include <QHBoxLayout>
#include <QVBoxLayout>
#include <QPushButton>
#include <QAbstractItemView>
#include <QLabel>
#include <QWidget>
#include <qstring.h>

#include "videowidget.h"
#include "configdialog.h"
#include <gst/gst.h>
#include <gst/video/videooverlay.h>
#include <glib.h>

QT_BEGIN_NAMESPACE
namespace Ui { class MainWindow; }
QT_END_NAMESPACE

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    /*MainWindow(QWidget *parent = nullptr);*/
    static MainWindow* getInstance();
    ~MainWindow();
    WId getWinId();
    WId getVideoWinID();
    void gstLaunch();

private slots:
    void openConfig();
    void openFile();

private:
    Ui::MainWindow *ui;
    static MainWindow* instance; // The only instance of MainWindow
    MainWindow(); // Private constructor

    QWidget* centralWidget = new QWidget(this);
    QVBoxLayout* centralLayout = new QVBoxLayout();

    GstElement* pipeline, * source, * sink;
    GstBus* bus;
    GstMessage* msg;
    GstStateChangeReturn ret;

    QMediaPlayer* player;
    QVideoWidget* vw;
    QProgressBar* bar;
    QSlider* slider;
    QPushButton* start;
    QPushButton* startFile;
    QPushButton* stop;

    ConfigDialog* cd;
    QString primaryIP;
    QString secondaryIP;
    QStringList list;

};

#endif // MAINWINDOW_H

My mainwindow is a singleton class and i'm using Qt Multimedia to display a video widget and directing the gstreamer output to that widget. I removed the autovideosink and it caused an exception, and then removed the width and height and that seemed to get rid of the exception. Now it looks like

void MainWindow::gstLaunch() // TODO: change return type to int to check successful pipeline launch
{
    gst_init(nullptr, nullptr);

    // [2] Prepare the pipeline
    //pipeline = gst_parse_launch(
    //    "playbin uri=https://www.freedesktop.org/software/gstreamer-sdk/data/media/sintel_trailer-480p.webm",
    //    NULL);
    pipeline = gst_parse_launch(
        "videotestsrc pattern=snow",
        NULL);

    // [4] Set video overlay
    gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(pipeline), vw->winId());

    // [5] Run the pipeline
    GstStateChangeReturn sret = gst_element_set_state(pipeline, GST_STATE_PLAYING);
    if (sret == GST_STATE_CHANGE_FAILURE)
    {
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        // Exit
        QTimer::singleShot(0, QApplication::activeWindow(), SLOT(quit()));
    }
}

However now it's giving me this error and still not showing the gstreamer output

QWindowsWindow::setGeometry: Unable to set geometry 148x38+789+440 (frame: 166x85+780+402) on QWidgetWindow/"ConfigDialogClassWindow" on "\\.\DISPLAY1". Resulting geometry: 319x125+789+440 (frame: 337x172+780+402) margins: 9, 38, 9, 9 minimum size: 255x100 MINMAXINFO(maxSize=POINT(x=0, y=0), maxpos=POINT(x=0, y=0), maxtrack=POINT(x=0, y=0), mintrack=POINT(x=337, y=172)))
'MRVideoStreamingTool.exe' (Win32): Loaded 'C:\gstreamer\1.0\msvc_x86_64\lib\gstreamer-1.0\gstvideotestsrc.dll'. Symbols loaded.

It looks like the pipeline is correct and it was able to load that resource? How do I fix the set geometry error? This is how i initialize my Qt Multimedia objects

player = new QMediaPlayer();
vw = new VideoWidget();
vw->resize(1280, 720);
player->setVideoOutput(vw);

[sdickerson]

Okay wait so it doesn't look like that's affecting it, that is for another window but that window still has functionality. So taking the autovideosink out doesn't display the gstreamer output at all

[SGaist]

Did you check the use of the qmlglsink ?

[sdickerson]

I have not, let me look at that and see if i can get it working

[sdickerson]

I'm a little confused on what I would need to do to get that working. I'm not currently using QML, i'm using QT Multimedia, so would i need to convert my entire project to QML? When I replace autovideosink with qmlglsink it still doesn't display the pipeline.

[sdickerson]

update: i got it to work correctly with this:

pipeline = gst_pipeline_new("xvoverlay");
src = gst_element_factory_make("videotestsrc", NULL);
sink = gst_element_factory_make("glimagesink", NULL);
gst_bin_add_many(GST_BIN(pipeline), src, sink, NULL);
gst_element_link(src, sink);
gst_debug_set_active(true);
gst_debug_set_default_threshold(GST_LEVEL_WARNING);

but would this just give me video and not audio?

[SGaist]

@sdickerson said in Gstreamer overlay set window handle QT6 not working:

I'm a little confused on what I would need to do to get that working. I'm not currently using QML, i'm using QT Multimedia, so would i need to convert my entire project to QML? When I replace autovideosink with qmlglsink it still doesn't display the pipeline.

No you don't have to. You can create a widget from the window of the sink. If memory serves well, there's a Qt example given with the plugin.

Are you sure the videosrc also provides audio ?

[sdickerson]

I needed to switch my project to Qt5 and tried using setMedia and gst:pipeline but that wasn't working for me either. Right now, the pipeline below works as it connects to the camera I need it to and opens the gstreamer output. However, it opens it in a new window and not the Qt Video Widget I am using. I know autovideosink ignores the window overlay, but I tried other sinks and just got a black screen. How could I get this to work? I tried glimagesink which worked for the videotestsrc but it did not work for this.

w->pipeline = gst_parse_launch(
        "udpsrc port=8000 ! application/x-rtp, encoding-name=H264, payload=109 ! rtph264depay ! h264parse ! queue ! 
        avdec_h264 ! videoconvert ! autovideosink",  NULL);

/* [4] Set video overlay */
 gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(w->pipeline), w->getVideoWinID());

/* [5] Run the pipeline */
 qDebug() << "------------------------------ Starting pipeline ------------------------------";
GstStateChangeReturn sret = gst_element_set_state(w->pipeline, GST_STATE_PLAYING);
 if (sret == GST_STATE_CHANGE_FAILURE)
{
        gst_element_set_state(w->pipeline, GST_STATE_NULL);
        gst_object_unref(w->pipeline);
}