Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. QWidget Flickering for frame by frame streaming of video
Forum Updated to NodeBB v4.3 + New Features

QWidget Flickering for frame by frame streaming of video

Scheduled Pinned Locked Moved Solved General and Desktop
qwidgetqpainteventflickeringvideopixmap
26 Posts 5 Posters 11.8k Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J.HilkJ J.Hilk

    @SilverSurfer
    hi, my guess would be, loadFromData takes longer than 34 ms, by higher resolutions. or the Drawing itself or the combination of both.

    I would suggest drawing more bitmaps beforehand in some kind of buffer, preferably threaded so you can do more than one drawing at once.

    THan access those premade bitmaps in your main thread, taking the oldest one from the buffer,for example a QList<QPixmaps>, and painting that one in your paintEvent.

    S Offline
    S Offline
    SilverSurfer
    wrote on last edited by SilverSurfer
    #5

    @J.Hilk
    Yes, loadFromData does take longer than 34ms(in fact 50ms at 4k resolution).

    This is for live streaming from IP Cameras, so I don't have the bitmaps beforehand.
    I do fetch bitmaps in a background thread and emit an signal (getImageData() is slot to this signal) to the main thread when they are available.

    The Flicker goes away if I reduce the fps, whereas loadFromData time remains the same. So is it really responsible for the flicker?

    I am also observing a lag which I think is because of loadFromData time.

    Is there any other way to draw an Image considering I have bitmap data in an unsigned char array ?

    J.HilkJ 1 Reply Last reply
    1
    • S SilverSurfer

      @J.Hilk
      Yes, loadFromData does take longer than 34ms(in fact 50ms at 4k resolution).

      This is for live streaming from IP Cameras, so I don't have the bitmaps beforehand.
      I do fetch bitmaps in a background thread and emit an signal (getImageData() is slot to this signal) to the main thread when they are available.

      The Flicker goes away if I reduce the fps, whereas loadFromData time remains the same. So is it really responsible for the flicker?

      I am also observing a lag which I think is because of loadFromData time.

      Is there any other way to draw an Image considering I have bitmap data in an unsigned char array ?

      J.HilkJ Offline
      J.HilkJ Offline
      J.Hilk
      Moderators
      wrote on last edited by
      #6

      @SilverSurfer
      I can think of 2 ways to do it.

      • don't stream directly, but delay the showing by a second or two, save your calculated images to a buffer and take from that to show in your application. Every so often you'll need to skip frames however

      • Split your image into 2, e.g. upper half and lower one, do the Pixmap modifications in 2 different threads and union that to images into one. This could bring you below the needed threshold.


      Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct


      Q: What's that?
      A: It's blue light.
      Q: What does it do?
      A: It turns blue.

      1 Reply Last reply
      1
      • S Offline
        S Offline
        SilverSurfer
        wrote on last edited by
        #7

        Is there any way I can avoid loadFromData() call, which I assume is doing a copy somewhere ?

        To rephrase my question:
        Can't I draw an Image from the existing buffer without a need for the copy? If yes , How?

        1 Reply Last reply
        0
        • SGaistS Offline
          SGaistS Offline
          SGaist
          Lifetime Qt Champion
          wrote on last edited by
          #8

          Hi,

          How are you getting the data in the first place ?

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          S 1 Reply Last reply
          0
          • SGaistS SGaist

            Hi,

            How are you getting the data in the first place ?

            S Offline
            S Offline
            SilverSurfer
            wrote on last edited by
            #9

            @SGaist
            By making a call to a server to fill the passed buffer.

            //time take by this depends size of buffer to be fetched
            //which may be more than 34 ms depending on size
            fetchBitmap(handle, cameraId,(uchar*)buffer , x_width, y_height);
            
            emit bufferAvailable(buffer,size); //calls getImageData(..) slot in the UI thread
            

            Thanks!

            1 Reply Last reply
            0
            • SGaistS Offline
              SGaistS Offline
              SGaist
              Lifetime Qt Champion
              wrote on last edited by
              #10

              What is that server ?

              What does it provide as API to retrieve images ?

              Interested in AI ? www.idiap.ch
              Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

              S 1 Reply Last reply
              0
              • SGaistS SGaist

                What is that server ?

                What does it provide as API to retrieve images ?

                S Offline
                S Offline
                SilverSurfer
                wrote on last edited by
                #11

                @SGaist
                It is a Video Management Server(ExacqVision Server to be precise).

                Yes, the API( fetchBitmap(...)) is provided by the server.
                So I don't think the data retrieval time can be reduced.

                What I would Ideally like to do is avoid any copying for this data.
                Which happens when I do pixmap->loadFromData(...) .

                1 Reply Last reply
                0
                • SGaistS Offline
                  SGaistS Offline
                  SGaist
                  Lifetime Qt Champion
                  wrote on last edited by
                  #12

                  Is their API documentation accessible somewhere ?

                  If you pass the buffer to the fetchBimap method, then you should be able to pass a QImage buffer pointer and in this case, the data will be copied directly on the QImage that you can then manipulate.

                  Interested in AI ? www.idiap.ch
                  Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                  S 1 Reply Last reply
                  1
                  • SGaistS SGaist

                    Is their API documentation accessible somewhere ?

                    If you pass the buffer to the fetchBimap method, then you should be able to pass a QImage buffer pointer and in this case, the data will be copied directly on the QImage that you can then manipulate.

                    S Offline
                    S Offline
                    SilverSurfer
                    wrote on last edited by SilverSurfer
                    #13

                    @SGaist
                    No the Documentation is not available publicly.

                    It take a uint8_t* as buffer argument.

                    //registered with server
                    Callback(event)
                    {
                        if(event == LIVE_FRAME_AVAILABLE)
                    {
                        frameSize = fetchBitmap(NULL); 
                        // Size fetched everytime as this can change with every frame if image is resized
                        vector.resize(framesize)
                        fetchBitmap(&vector[0]);
                        emit bufferAvailable(&vector[0],frameSize);
                    }
                    }
                    

                    I am not sure how can I use QImage pointer here.

                    1 Reply Last reply
                    0
                    • SGaistS Offline
                      SGaistS Offline
                      SGaist
                      Lifetime Qt Champion
                      wrote on last edited by
                      #14

                      QImage::bits.

                      Make the QImage the correct format and size.

                      Interested in AI ? www.idiap.ch
                      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                      S 1 Reply Last reply
                      1
                      • SGaistS SGaist

                        QImage::bits.

                        Make the QImage the correct format and size.

                        S Offline
                        S Offline
                        SilverSurfer
                        wrote on last edited by
                        #15

                        @SGaist
                        Tried using
                        QImage::QImage(uchar * data, int width, int height, Format format, QImageCleanupFunction cleanupFunction = 0, void * cleanupInfo = 0)

                        data = pixel data , updated by fetchBitmap(..) function above.
                        format = RGB888
                        where as per documentation it does not do a deep copy. So this helps.

                        But my image has R and B planes swapped.
                        Can't seem to find the appropriate format .

                        P.S: I am on a Windows machine.

                        Thanks,
                        Kshitij Mayekar

                        1 Reply Last reply
                        0
                        • SGaistS Offline
                          SGaistS Offline
                          SGaist
                          Lifetime Qt Champion
                          wrote on last edited by
                          #16

                          See QImage::rgbSwapped

                          Interested in AI ? www.idiap.ch
                          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                          S 1 Reply Last reply
                          1
                          • SGaistS SGaist

                            See QImage::rgbSwapped

                            S Offline
                            S Offline
                            SilverSurfer
                            wrote on last edited by
                            #17

                            @SGaist
                            QImage::rgbSwapped results in creating new QImage with is equivalent to deep copy. So It does not help. Is there any other way to dislaying the RGB planes correctly ?

                            Note: Whenever I write the contents of the buffer to a file with appropriate headers I could see the Image correctly when I open the Image through windows photo viewer.

                            Original Problem Statement : Display Byte Buffer(uchar*) to a QWidget without the need to deep Copy .

                            1 Reply Last reply
                            0
                            • SGaistS Offline
                              SGaistS Offline
                              SGaist
                              Lifetime Qt Champion
                              wrote on last edited by
                              #18

                              Where are these frames coming from ?

                              Interested in AI ? www.idiap.ch
                              Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

                              S 1 Reply Last reply
                              0
                              • SGaistS SGaist

                                Where are these frames coming from ?

                                S Offline
                                S Offline
                                SilverSurfer
                                wrote on last edited by
                                #19

                                @SGaist

                                These frames are received from a server.

                                //Api that talks to server
                                fetchBitmap(&vector[0]);
                                

                                I do not have access to server code.

                                P.S: Server Machine is Linux and Client is Windows . Could this be the reason for RGB /BGR mismatch ?

                                mrjjM 1 Reply Last reply
                                0
                                • W Offline
                                  W Offline
                                  wrosecrans
                                  wrote on last edited by
                                  #20

                                  Is there any chance of the paint event being triggered by the system at a moment when the pixmap isn't fully populated? If that is possible, you would get flickering regardless of whether the load function is faster or slower than real time.

                                  1 Reply Last reply
                                  1
                                  • S SilverSurfer

                                    @SGaist

                                    These frames are received from a server.

                                    //Api that talks to server
                                    fetchBitmap(&vector[0]);
                                    

                                    I do not have access to server code.

                                    P.S: Server Machine is Linux and Client is Windows . Could this be the reason for RGB /BGR mismatch ?

                                    mrjjM Offline
                                    mrjjM Offline
                                    mrjj
                                    Lifetime Qt Champion
                                    wrote on last edited by
                                    #21

                                    @SilverSurfer said in QWidget Flickering for frame by frame streaming of video:

                                    Could this be the reason for RGB /BGR mismatch ?

                                    Well there is always Endianness but if using same sort of CPU in both ends, its likely not related to that.

                                    S 1 Reply Last reply
                                    0
                                    • mrjjM mrjj

                                      @SilverSurfer said in QWidget Flickering for frame by frame streaming of video:

                                      Could this be the reason for RGB /BGR mismatch ?

                                      Well there is always Endianness but if using same sort of CPU in both ends, its likely not related to that.

                                      S Offline
                                      S Offline
                                      SilverSurfer
                                      wrote on last edited by
                                      #22

                                      @mrjj
                                      Both machines are little endian.

                                      mrjjM 1 Reply Last reply
                                      0
                                      • S SilverSurfer

                                        @mrjj
                                        Both machines are little endian.

                                        mrjjM Offline
                                        mrjjM Offline
                                        mrjj
                                        Lifetime Qt Champion
                                        wrote on last edited by
                                        #23

                                        @SilverSurfer
                                        Ok. so most likely they
                                        are just using openCV and send as BGR888

                                        So i dont see any way to convert to supported format without copy.

                                        W 1 Reply Last reply
                                        0
                                        • mrjjM mrjj

                                          @SilverSurfer
                                          Ok. so most likely they
                                          are just using openCV and send as BGR888

                                          So i dont see any way to convert to supported format without copy.

                                          W Offline
                                          W Offline
                                          wrosecrans
                                          wrote on last edited by
                                          #24

                                          @mrjj If you use a shader in a QOpenGLWidget, you can swizzle the r/g/b around arbitrarily when you do the drawing into the frame buffer, which would avoid an extra copy step to normalize the channel order.

                                          S 1 Reply Last reply
                                          1

                                          • Login

                                          • Login or register to search.
                                          • First post
                                            Last post
                                          0
                                          • Categories
                                          • Recent
                                          • Tags
                                          • Popular
                                          • Users
                                          • Groups
                                          • Search
                                          • Get Qt Extensions
                                          • Unsolved