QWidget Flickering for frame by frame streaming of video

[J.Hilk]

@SilverSurfer
I can think of 2 ways to do it.

• don't stream directly, but delay the showing by a second or two, save your calculated images to a buffer and take from that to show in your application. Every so often you'll need to skip frames however

• Split your image into 2, e.g. upper half and lower one, do the Pixmap modifications in 2 different threads and union that to images into one. This could bring you below the needed threshold.

[SilverSurfer]

Is there any way I can avoid loadFromData() call, which I assume is doing a copy somewhere ?

To rephrase my question:
Can't I draw an Image from the existing buffer without a need for the copy? If yes , How?

[SGaist]

Hi,

How are you getting the data in the first place ?

[SilverSurfer]

@SGaist
By making a call to a server to fill the passed buffer.

//time take by this depends size of buffer to be fetched
//which may be more than 34 ms depending on size
fetchBitmap(handle, cameraId,(uchar*)buffer , x_width, y_height);

emit bufferAvailable(buffer,size); //calls getImageData(..) slot in the UI thread

Thanks!

[SGaist]

What is that server ?

What does it provide as API to retrieve images ?

[SilverSurfer]

@SGaist
It is a Video Management Server(ExacqVision Server to be precise).

Yes, the API( fetchBitmap(...)) is provided by the server.
So I don't think the data retrieval time can be reduced.

What I would Ideally like to do is avoid any copying for this data.
Which happens when I do pixmap->loadFromData(...) .

[SGaist]

Is their API documentation accessible somewhere ?

If you pass the buffer to the fetchBimap method, then you should be able to pass a QImage buffer pointer and in this case, the data will be copied directly on the QImage that you can then manipulate.

[SilverSurfer]

@SGaist
No the Documentation is not available publicly.

It take a uint8_t* as buffer argument.

//registered with server
Callback(event)
{
    if(event == LIVE_FRAME_AVAILABLE)
{
    frameSize = fetchBitmap(NULL); 
    // Size fetched everytime as this can change with every frame if image is resized
    vector.resize(framesize)
    fetchBitmap(&vector[0]);
    emit bufferAvailable(&vector[0],frameSize);
}
}

I am not sure how can I use QImage pointer here.

[SGaist]

QImage::bits.

Make the QImage the correct format and size.

[SilverSurfer]

@SGaist
Tried using
QImage::QImage(uchar * data, int width, int height, Format format, QImageCleanupFunction cleanupFunction = 0, void * cleanupInfo = 0)

data = pixel data , updated by fetchBitmap(..) function above.
format = RGB888
where as per documentation it does not do a deep copy. So this helps.

But my image has R and B planes swapped.
Can't seem to find the appropriate format .

P.S: I am on a Windows machine.

Thanks,
Kshitij Mayekar

[SGaist]

See QImage::rgbSwapped

[SilverSurfer]

@SGaist
QImage::rgbSwapped results in creating new QImage with is equivalent to deep copy. So It does not help. Is there any other way to dislaying the RGB planes correctly ?

Note: Whenever I write the contents of the buffer to a file with appropriate headers I could see the Image correctly when I open the Image through windows photo viewer.

Original Problem Statement : Display Byte Buffer(uchar*) to a QWidget without the need to deep Copy .

[SGaist]

Where are these frames coming from ?

[SilverSurfer]

@SGaist

These frames are received from a server.

//Api that talks to server
fetchBitmap(&vector[0]);

I do not have access to server code.

P.S: Server Machine is Linux and Client is Windows . Could this be the reason for RGB /BGR mismatch ?

[wrosecrans]

Is there any chance of the paint event being triggered by the system at a moment when the pixmap isn't fully populated? If that is possible, you would get flickering regardless of whether the load function is faster or slower than real time.

[mrjj]

@SilverSurfer said in QWidget Flickering for frame by frame streaming of video:

Could this be the reason for RGB /BGR mismatch ?

Well there is always Endianness but if using same sort of CPU in both ends, its likely not related to that.

[SilverSurfer]

@mrjj
Both machines are little endian.

[mrjj]

@SilverSurfer
Ok. so most likely they
are just using openCV and send as BGR888

So i dont see any way to convert to supported format without copy.

[wrosecrans]

@mrjj If you use a shader in a QOpenGLWidget, you can swizzle the r/g/b around arbitrarily when you do the drawing into the frame buffer, which would avoid an extra copy step to normalize the channel order.

[SilverSurfer]

@wrosecrans said in QWidget Flickering for frame by frame streaming of video:

you can swizzle the r/g/b around arbitrarily when you do the drawing into the frame buffer

Could you provide sample code for the same ?