Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. QML and Qt Quick
  4. How to use QAbstractVideoFilter to process frames from camera
QtWS25 Last Chance

How to use QAbstractVideoFilter to process frames from camera

Scheduled Pinned Locked Moved QML and Qt Quick
camerafilterframeqabstractvideof
27 Posts 10 Posters 18.8k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • T theshadowx

    @medyakovvit Hi, thanks for responding, I hope everything is fine for you.

    if my camera takes video in RGB32 it would be easier as I can create QImage from the edge matrix and then create a QVideoFrame from QImage.:

     ///cv::Mat to QImage
     QImage ImgDest(edges.data, edges.cols, edges.rows, edges.step, QImage::Format_RGB888);
     QVideoFrame *output = new QVideoFrame(ImgDest);
    

    And then I would return the output.

    The problem (From what I understood) is that my camera generates images with YUV color scheme. which means that QVideoSurfaceFormat is in YUV, so if I use RGB32 format it won't work.

    Unfortunately QImage doesn't support YUV format. :(

    M Offline
    M Offline
    medyakovvit
    wrote on last edited by
    #15

    @theshadowx So, if correctly understood this YUV420 and YUV format, Y - luma component, U and V - color component. And in single frame Ys go first, then Us and Vs. So if you need grayscale image, you can work with Ys and ignore Us and Vs. Maybe you can try:

    QVideoFrame YUVFilterRunnable::run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, QVideoFilterRunnable::RunFlags flags)
    {
        if (!input->isValid())
            return *input;
    
        input->map(QAbstractVideoBuffer::ReadWrite);
    
        this->deleteColorComponentFromYUV(input);
        cv::Mat mat(input->height(),input->width(), CV_8UC1, input->bits()); // create grayscale mat with input's Ys only and avoid additional color conversion and copying
    
        cv::GaussianBlur(mat, mat, Size(7,7), 1.5, 1.5);
        cv::Canny(mat, mat, 0, 30, 3);
    
        input->unmap();
        return *input;
    }
    
    void YUVFilterRunnable::deleteColorComponentFromYUV(QVideoFrame *input)
    {
        // Assign 0 to Us and Vs
        int firstU = input->width()*input->height(); // if i correctly understand YUV420
        int lastV = input->width()*input->height() + input->width()*input->height()/4*2;
        uchar* inputBits = input->bits();
    
        for (int i=firstU; i<lastV; i++)
            inputBits[i] = 0;
    }
    
    T 1 Reply Last reply
    0
    • M medyakovvit

      @theshadowx So, if correctly understood this YUV420 and YUV format, Y - luma component, U and V - color component. And in single frame Ys go first, then Us and Vs. So if you need grayscale image, you can work with Ys and ignore Us and Vs. Maybe you can try:

      QVideoFrame YUVFilterRunnable::run(QVideoFrame *input, const QVideoSurfaceFormat &surfaceFormat, QVideoFilterRunnable::RunFlags flags)
      {
          if (!input->isValid())
              return *input;
      
          input->map(QAbstractVideoBuffer::ReadWrite);
      
          this->deleteColorComponentFromYUV(input);
          cv::Mat mat(input->height(),input->width(), CV_8UC1, input->bits()); // create grayscale mat with input's Ys only and avoid additional color conversion and copying
      
          cv::GaussianBlur(mat, mat, Size(7,7), 1.5, 1.5);
          cv::Canny(mat, mat, 0, 30, 3);
      
          input->unmap();
          return *input;
      }
      
      void YUVFilterRunnable::deleteColorComponentFromYUV(QVideoFrame *input)
      {
          // Assign 0 to Us and Vs
          int firstU = input->width()*input->height(); // if i correctly understand YUV420
          int lastV = input->width()*input->height() + input->width()*input->height()/4*2;
          uchar* inputBits = input->bits();
      
          for (int i=firstU; i<lastV; i++)
              inputBits[i] = 0;
      }
      
      T Offline
      T Offline
      theshadowx
      wrote on last edited by theshadowx
      #16

      @medyakovvit

      Yes that's right, there is one porblem in the code is that with you code I get a greenscale. So to convert to grayscale the solution is that instead of 0 for chroma, it should be 127 (http://stackoverflow.com/a/20609599/2775917) :

      void YUVFilterRunnable::deleteColorComponentFromYUV(QVideoFrame *input)
      {
          // Assign 0 to Us and Vs
          int firstU = input->width()*input->height(); // if i correctly understand YUV420
          int lastV = input->width()*input->height() + input->width()*input->height()/4*2;
          uchar* inputBits = input->bits();
      
          for (int i=firstU; i<lastV; i++)
              inputBits[i] = 127;
      }
      

      Thanks a lot for your help

      M 1 Reply Last reply
      0
      • T theshadowx

        @medyakovvit

        Yes that's right, there is one porblem in the code is that with you code I get a greenscale. So to convert to grayscale the solution is that instead of 0 for chroma, it should be 127 (http://stackoverflow.com/a/20609599/2775917) :

        void YUVFilterRunnable::deleteColorComponentFromYUV(QVideoFrame *input)
        {
            // Assign 0 to Us and Vs
            int firstU = input->width()*input->height(); // if i correctly understand YUV420
            int lastV = input->width()*input->height() + input->width()*input->height()/4*2;
            uchar* inputBits = input->bits();
        
            for (int i=firstU; i<lastV; i++)
                inputBits[i] = 127;
        }
        

        Thanks a lot for your help

        M Offline
        M Offline
        medyakovvit
        wrote on last edited by
        #17

        @theshadowx I'm glad to help

        1 Reply Last reply
        0
        • C Offline
          C Offline
          cs_a994
          wrote on last edited by
          #18

          Hello!

          Is it possible to use the QVideoFilterRunnuble without QML?

          As an example I would like to embed it into Camera example ( http://doc.qt.io/qt-5/qtmultimediawidgets-camera-example.html ) to be able to change data format.

          Thank you.

          M 1 Reply Last reply
          0
          • C cs_a994

            Hello!

            Is it possible to use the QVideoFilterRunnuble without QML?

            As an example I would like to embed it into Camera example ( http://doc.qt.io/qt-5/qtmultimediawidgets-camera-example.html ) to be able to change data format.

            Thank you.

            M Offline
            M Offline
            medyakovvit
            wrote on last edited by
            #19

            @cs_a994 Don't think so. But you can try to subclass QAbstractVideoSurface.

            C 1 Reply Last reply
            0
            • M medyakovvit

              @cs_a994 Don't think so. But you can try to subclass QAbstractVideoSurface.

              C Offline
              C Offline
              cs_a994
              wrote on last edited by
              #20

              @medyakovvit

              Thank you. I know about this way. But use of filters seams to me more neat.

              I tryed to use filters with QML ( in the Camera example ) and did not yet get a result.

              M T 2 Replies Last reply
              0
              • C cs_a994

                @medyakovvit

                Thank you. I know about this way. But use of filters seams to me more neat.

                I tryed to use filters with QML ( in the Camera example ) and did not yet get a result.

                M Offline
                M Offline
                medyakovvit
                wrote on last edited by
                #21

                @cs_a994
                OK. I f you have a questions about qml camera and filters, I will try to help you.

                1 Reply Last reply
                0
                • C cs_a994

                  @medyakovvit

                  Thank you. I know about this way. But use of filters seams to me more neat.

                  I tryed to use filters with QML ( in the Camera example ) and did not yet get a result.

                  T Offline
                  T Offline
                  theshadowx
                  wrote on last edited by
                  #22

                  @cs_a994

                  I have put a QtQuick app in github using QAbstractVideoFilter with OpenCV.

                  I hope it helps you.

                  Cheers

                  C 1 Reply Last reply
                  0
                  • T theshadowx

                    @cs_a994

                    I have put a QtQuick app in github using QAbstractVideoFilter with OpenCV.

                    I hope it helps you.

                    Cheers

                    C Offline
                    C Offline
                    cs_a994
                    wrote on last edited by
                    #23

                    @theshadowx

                    Thank you very much. I'll study it soon. I hope it will help.

                    1 Reply Last reply
                    0
                    • M Offline
                      M Offline
                      mikeitexpert
                      wrote on last edited by mikeitexpert
                      #24

                      @theshadowx

                      I get the below error if I try the canny code from git hub. (https://github.com/theshadowx/Qt_OpenCV)

                      I don't know much about mql but I know QT intermediately.

                      Thank you

                      01:45:55: Starting /home/mike/Downloads/Qt_OpenCV-master/QtQuick/build-CannyQml-Desktop_Qt_5_12_0_GCC_64bit-Debug/CannyQml...
                      QML debugging is enabled. Only use this in a safe environment.

                      (CannyQml:24471): GStreamer-CRITICAL **: 01:45:57.866: write map requested on non-writable buffer
                      01:45:57: The program has unexpectedly finished.
                      01:45:57: The process was ended forcefully.
                      01:45:58: /home/mike/Downloads/Qt_OpenCV-master/QtQuick/build-CannyQml-Desktop_Qt_5_12_0_GCC_64bit-Debug/CannyQml crashed.

                      T 1 Reply Last reply
                      0
                      • M mikeitexpert

                        @theshadowx

                        I get the below error if I try the canny code from git hub. (https://github.com/theshadowx/Qt_OpenCV)

                        I don't know much about mql but I know QT intermediately.

                        Thank you

                        01:45:55: Starting /home/mike/Downloads/Qt_OpenCV-master/QtQuick/build-CannyQml-Desktop_Qt_5_12_0_GCC_64bit-Debug/CannyQml...
                        QML debugging is enabled. Only use this in a safe environment.

                        (CannyQml:24471): GStreamer-CRITICAL **: 01:45:57.866: write map requested on non-writable buffer
                        01:45:57: The program has unexpectedly finished.
                        01:45:57: The process was ended forcefully.
                        01:45:58: /home/mike/Downloads/Qt_OpenCV-master/QtQuick/build-CannyQml-Desktop_Qt_5_12_0_GCC_64bit-Debug/CannyQml crashed.

                        T Offline
                        T Offline
                        theshadowx
                        wrote on last edited by
                        #25

                        @mikeitexpert
                        Could you add this issue in https://github.com/theshadowx/Qt_OpenCV/issues

                        Thanks

                        1 Reply Last reply
                        0
                        • M Offline
                          M Offline
                          mikeitexpert
                          wrote on last edited by
                          #26

                          here: https://github.com/theshadowx/Qt_OpenCV/issues/5

                          Is there any way I can convert qml to c++ code just to get an idea how your methods works in pure C++?

                          Thank you

                          1 Reply Last reply
                          0
                          • M Offline
                            M Offline
                            mikeitexpert
                            wrote on last edited by
                            #27

                            Please let me know if there is any update on this.

                            Mike

                            1 Reply Last reply
                            0

                            • Login

                            • Login or register to search.
                            • First post
                              Last post
                            0
                            • Categories
                            • Recent
                            • Tags
                            • Popular
                            • Users
                            • Groups
                            • Search
                            • Get Qt Extensions
                            • Unsolved