Preprocessing images from Camera before Viewfinder

[sergboec]

Hello!
So i have a camera and i want to do these simple steps

1. preprocess every frame from camera

2. display preprocessed

3. paint some lines and text over it

i guess i should use QGraphicsVideoItem for diplaying and draw over it
and QAbstractVideoFilter can help me to process each frame

But i can't find any way to combine these two

[SGaist]

Hi,

What kind of pre-processing do you have in mind ?

QtQuick might be a better choice depending on what you want to render.

[sergboec]

Yeah, I switched to QML
For now i have a problem when format of QVideoFrame is YUYV and QImage doesn't support it.

I managed to load image directly from QVideoFrame into opencv::Mat but i can't return proper QVideoFrame from
my implementation of

QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {

because as far as i understand I need to return QVideoFrame in YUYV but i can't create it cause QImage doesn't support it

[SGaist]

What operation do you want to apply to the video frame ?

[sergboec]

Right now i have a very strange situation
ex1:
please look at code

class MyFilterRunnable : public QVideoFilterRunnable {
public:
    QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {

        if (input->isValid()) {
            //reading from QVideoFrame
            QVideoFrame cloneFrame(*input);
            cloneFrame.map(QAbstractVideoBuffer::ReadOnly);

            cv::Mat mat(cloneFrame.height(), cloneFrame.width(), CV_8UC2, cloneFrame.bits(), cloneFrame.bytesPerLine());

            cv::Mat converted_image = cv::Mat(mat);
            //converting to RGB
            cvtColor(mat, converted_image, CV_YUV2RGB_YUYV);
            //drawing dummy circle just as an example
            cv::circle(converted_image, cv::Point(100, 100), 35, cv::Scalar(0,0,255), 2, CV_AA );
            //constructing image
            QImage imageToReturn = QImage((uchar*)converted_image.data, converted_image.cols, converted_image.rows, QImage::Format_RGB888);

            //closing video frame mapping
            cloneFrame.unmap();
            imageToReturn.save("file.bmp","bmp",100);

            auto videoFrameToReturn = QVideoFrame(imageToReturn);

            qDebug() << v.isValid(); // always prints "true"
            return videoFrameToReturn;
}

So as you can see i'm creating an cv::Mat object drawing plain circle and packing it into QImage.
After that i'm creating QVideFrame from QImage and **the view in QML is grey **
But!
In code number 2
Everything shows as it should be

class MyFilterRunnable : public QVideoFilterRunnable {
public:
    QVideoFrame run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, RunFlags flags) {

        if (input->isValid()) {

//            tmp.save("file.bmp","bmp",100);

            QImage imageToReturn = QImage("file.bmp","bmp");

            auto videoFrameToReturn = QVideoFrame(imageToReturn);
}

I even can write to file my QImage, instantly read it into another QImage and everything works

[SGaist]

In your second code, you don't do anything to input so I'm not sure what you are expecting there.

[sergboec]

Ok.
I have found a solution

QImage imageToReturn = QImage((uchar*)converted_image.data, converted_image.cols, converted_image.rows, QImage::Format_RGB888);

imageToReturn.convertToFormat(QImage::Format_RGB32);

I have found that when i started to compare metadata between QImage constructed from cv::Mat data and QImage constructed as in second example