How to get QImage from QVideoSink in Android ?

[squareroot13]

Hello, i made the code, which works perfectly in Windows, but doesn't work in Android. I need an advice how to get QImage as preview image in QMediaPlayer.
My code is below:

                pplayer->stop();
                pplayer->setSource(url);
                int attempts = 25;
                bool isBroken = (pplayer->mediaStatus() == QMediaPlayer::LoadedMedia) &&
                                (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                while(isBroken && pplayer) {
                    QThread::msleep(20);
                    if (!attempts)
                        break;
                    attempts--;
                    isBroken = (pplayer->mediaStatus() == QMediaPlayer::LoadedMedia) &&
                                (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                }
                if (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia) {
                    qWarning() << "No preview for:" << abspath << " QMediaPlayer::InvalidMedia";
                    break;
                }
                m_alreadyFrame = false;
                qWarning() << "before wait:";
                pplayer->pause();
                QCoreApplication::processEvents();
                QThread::msleep(10);
                attempts = 20;
                isBroken = (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                while (!m_alreadyFrame && !isBroken && attempts) {
                    QCoreApplication::processEvents();
                    QThread::msleep(10);
                    isBroken = (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                    attempts--;
                }
                qWarning() << "after wait:";
                QCoreApplication::processEvents();
                if (m_alreadyFrame) {
                    frame = psink->videoFrame();
                    frame.map(QVideoFrame::ReadOnly);
                    image = frame.toImage().copy();
                    qWarning() << "format:" << image.format();
                    frame.unmap();
                    if (image.format() == QImage::Format_Invalid) {
                        qWarning() << "QImage::Format_Invalid";
                        isBroken = true;
                    }
                }
                pplayer->stop();
                m_alreadyFrame = false;
                if (isBroken || !attempts) {
                    qWarning() << "No preview for:" << abspath ;
                    qWarning() << "attempts:" << attempts << " isBroken:" << isBroken;
                    break;
                }
                //setState(TPrevStates::eIdle);
                //break;
                QString key = fileName;
                qWarning() << "key is:" << key << " width is:" << image.width();

    connect(psink.get(), &QVideoSink::videoFrameChanged, this,
                         [&](const QVideoFrame & _frame){
                if (!_frame.isValid()) {
                    qWarning() << "frame is not valid";
                    return;
                }
                m_alreadyFrame = true;
    });

[SGaist]

Hi,

Which version of Qt are you using ?

[squareroot13]

@SGaist 6.9.0

[SGaist]

You are not checking that the mapping is successful. That would be a first step to identify the issue.
Also, which version of Android are you using ?
What kind of video are you playing ?

[squareroot13]

@SGaist Android 11. mp4 file and h264 codec - most popular case

[squareroot13]

@SGaist I did a map check. result is false. (for android only)
Where i'm wrong ?

[squareroot13]

I have changed my code in order it was more correct.

                pplayer->stop();
                pplayer->setSource(url);
                int attempts = 25;
                bool isBroken = (pplayer->mediaStatus() == QMediaPlayer::LoadedMedia) &&
                                (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                while(isBroken && pplayer) {
                    QThread::msleep(20);
                    if (!attempts)
                        break;
                    attempts--;
                    isBroken = (pplayer->mediaStatus() == QMediaPlayer::LoadedMedia) &&
                                (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                }
                if (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia) {
                    qWarning() << "No preview for:" << abspath << " QMediaPlayer::InvalidMedia";
                    break;
                }
                m_alreadyFrame = false;
                qWarning() << "before wait:";
                pplayer->pause();
                QCoreApplication::processEvents();
                QThread::msleep(10);
                attempts = 20;
                isBroken = (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                while (!m_alreadyFrame && !isBroken && attempts) {
                    QCoreApplication::processEvents();
                    QThread::msleep(10);
                    isBroken = (pplayer->mediaStatus() == QMediaPlayer::InvalidMedia);
                    attempts--;
                }
                qWarning() << "after wait:";
                QCoreApplication::processEvents();
                if (m_alreadyFrame) {
                    const QVideoFrame& vf = psink->videoFrame();
                    if (psink->videoFrame().map(QVideoFrame::ReadOnly)) {
                        image = psink->videoFrame().toImage();
                        psink->videoFrame().unmap();
                    }
                    else {
                        qWarning() << "QVideoFrame psink is not mapped";
                        break;
                    }
                    if (image.format() == QImage::Format_Invalid) {
                        qWarning() << "QImage::Format_Invalid";
                        isBroken = true;
                    }
                }
                pplayer->stop();
                m_alreadyFrame = false;
                if (isBroken || !attempts) {
                    qWarning() << "No preview for:" << abspath ;
                    qWarning() << "attempts:" << attempts << " isBroken:" << isBroken;
                    break;
                }
                //setState(TPrevStates::eIdle);
                //break;
                QString key = fileName;
                qWarning() << "key is:" << key << " width is:" << image.width();

But i have the same result as before: success on windows and a failure on Android. Windows successfully mapped current frame.
Chunk of log on Android device is below:

....
W/default (24221): after wait:
W/default (24221): Error transferring the data to system memory: -38
W/default (24221): QVideoFrame psink is not mapped

W/default (24221): start of processing for: "Trie Data Structure and Aho-Corasick Algorithm – Trie-дерево и Алгоритм Ахо-Корасик.mp4"
W/default (24221): before wait:
W/default (24221): frame is not valid
W/default (24221): after wait:
W/default (24221): Error transferring the data to system memory: -38
W/default (24221): QVideoFrame psink is not mapped

W/default (24221): start of processing for: "videoplayback.mp4"
W/default (24221): before wait:
W/default (24221): frame is not valid
W/default (24221): after wait:
W/default (24221): Error transferring the data to system memory: -38
W/default (24221): QVideoFrame psink is not mapped
W/default (24221): return to Idle

[squareroot13]

Partial solution is found: https://bugreports.qt.io/browse/QTBUG-138915