QVideoProbe has bad performance under windows

draekster

Hello!

I'm new to Qt (and c++...) and have a problem with my project.
I use QMediaPlayer to play a video file. This video is shown in a QGraphicsScene. There I can see the video.

From this video I capture a QFrame with QVideoProbe and display this frame in another QGraphicsScene.

When I run this program under Linux all works perfekt. I see the video and the frame.
But under Windows the performance is very bad. When I have luck I see some frames and the program does not crash. But the most time the programm chrashes after a few seconds.

Did I something wrong?

My System Linux:
OS: Manjaro Linux 64bit
Compiler: GCC
QT: 5.4.1

My System Windows
OS: WIndows 7 64bit
Compiler: msvc2012
QT: 5.4.1

On both systems I use QTCreator.

mainwindow.cpp

#include "mainwindow.h"
#include "ui_mainwindow.h"

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    player = new QMediaPlayer;
    video = new QGraphicsVideoItem;
    monitor = new QImage;
    sceneVideo = new QGraphicsScene(this);
    sceneMonitor = new QGraphicsScene(this);
    videoProbe = new QVideoProbe(this);

    ui->graphicsViewVideo->setScene(sceneVideo);
    ui->graphicsViewMonitor->setScene(sceneMonitor);

    video->setSize(QSizeF(ui->graphicsViewVideo->width(), ui->graphicsViewVideo->height()));

    sceneVideo->addItem(video);
    player->setVideoOutput(video);

    qDebug() << "Frames werden abgegriffen: " << videoProbe->setSource(player);

    player->setMuted(true);
    time = new QTime(0,0,0,0);

    connect(player, &QMediaPlayer::positionChanged, this, &MainWindow::on_positionChanged);
    connect(player, &QMediaPlayer::durationChanged, this, &MainWindow::on_durationChanged);
    connect(videoProbe, SIGNAL(videoFrameProbed(QVideoFrame)), this, SLOT(on_frameCaptured(QVideoFrame)));
}

MainWindow::~MainWindow()
{
    delete video;
    delete monitor;
    delete player;
    delete sceneVideo;
    delete ui;
}

void MainWindow::on_actionLoadVideo_triggered()
{
    QString filename = QFileDialog::getOpenFileName(this, tr("Öffne Video"), ".",tr("Video Dateien (*.*)"));
    if (!filename.isEmpty()){
        player->setMedia(QUrl::fromLocalFile(filename));
    }
}

void MainWindow::on_videoProgress_sliderMoved(int position)
{
    player->setPosition(position);
    ui->videoProgress->setValue(position);
}

void MainWindow::on_pushPlayPause_clicked()
{
    if((player->state() == QMediaPlayer::StoppedState) || (player->state() == QMediaPlayer::PausedState)){
        player->play();
    }
    else{
        player->pause();
    }
}

void MainWindow::on_pushButtonStop_clicked()
{
    player->stop();
}

void MainWindow::on_durationChanged(int dur){
    ui->videoProgress->setMaximum(dur);
    ui->labelDuration->setText(time->addMSecs(dur).toString());
}

void MainWindow::on_positionChanged(int pos){
    ui->videoProgress->setValue(pos);
    ui->labelCurrentPosition->setText(time->addMSecs(pos).toString());
}

void MainWindow::on_frameCaptured(const QVideoFrame &frame){
    QVideoFrame f(frame);
    f.map(QAbstractVideoBuffer::ReadOnly);

    monitor = new QImage(f.bits(), f.width(), f.height(), QVideoFrame::imageFormatFromPixelFormat(f.pixelFormat()));

    sceneMonitor->clear();
    sceneMonitor->addPixmap(QPixmap::fromImage(*monitor).scaled(ui->graphicsViewVideo->width(),ui->graphicsViewVideo->height(),Qt::KeepAspectRatio));

    f.unmap();
}

void MainWindow::resizeEvent(QResizeEvent *) {
    video->setSize(QSizeF(ui->graphicsViewVideo->width(), ui->graphicsViewVideo->height()));
    if(!QPixmap::fromImage(*monitor).isNull()){
        sceneMonitor->clear();
        sceneMonitor->addPixmap(QPixmap::fromImage(*monitor).scaled(ui->graphicsViewVideo->width(),ui->graphicsViewVideo->height(),Qt::KeepAspectRatio));
    }
}

Workaround in Post 6.

SGaist

Hi and welcome to devnet,

The performance of the Qt Multimedia backends can vary from one platform to the other.

One thing that is sure is that you have a memory leak in on_frameCaptured. You allocate a new QImage each time it's called but you never release it. You don't need heap allocation when dealing with QImage. It's one of Qt shared classes and it uses heap allocation internally for the data part.

draekster

Thank you for your reply and it seems that I solved the memory leak. (Thank you for pointing that out.)

But I can't believe that my player is this slow...
Is there a way to speed this thing?

Maybe with other codecs or so? (I've tried to install other codecs... with no improvement.)
Or do I have to enable ffmpeg or something? (I didn't find something about it.)

SGaist

What format is giving you trouble ?

draekster

My testfile is MP4 from my Canon camera. But I also tried it with an example WMV file from Windows.

draekster

Ok I found a workaround for my problem.

The method on_frameCaptured() starts a thread now.
This thread do some other tasks and sleeps after that for a few milliseconds. While the thread is running the method on_frameCaptured() does nothing.

void MainWindow::on_frameCaptured(const QVideoFrame &frame){
     if(thread is not running){
          do something and
          start thread
     }
}

Another workaround is to block the signals for the videoProbe while the method is working.

void MainWindow::on_frameCaptured(const QVideoFrame &frame){
     videoProbe block signals
     do something
     videoProbe unblock signals
}

But I think in my case the thread "solution" is the better one.

Another realy annoying problem was the format of the QImage...
In my case Linux use QImage::Format_RGB32.
Windows uses QImage::Format_RGB888.
And this with the same video file...

So I have to convert the QImages on Windows to QImage::Format_RGB32.

The performance under Windows is still bad but i can work with that for now.

SGaist

Depending on what you need to do, shouldn't you rather use a QQueue for the frames and let the thread run as fast as it can ?

draekster

I don't need every frame.
But thank you for the tip I will give it a try later.