QVideoFrame handleType is NoHandle for camera frames on Android
-
Hi,
I'm trying to get camera frames from a QCamera to use them as a texture in a shader. Using Qt 6.6.1.
I made a simple test application, where I instantiate a QCaptureSession and its QCamera, and set the capture session video sink to a class derived from QVideoSink to get the QVideoFrame whenever the videoFrameChanged signal is emitted.
I checked the frame type with handleType(), and it appears to be set to QVideoFrame::NoHandle. I tested it on both Desktop, with an Android emulator, and with a real Android device.
I was expecting a QVideoFrame::RhiTextureHandle on Android, as with previous Qt versions it seemed to have been the case, returning it as a "GLTextureHandle" ?
https://forum.qt.io/topic/103430/qvideoframe-map-is-false-in-android
https://forum.qt.io/topic/55810/qvideoframe-problem-with-mapping-memory-from-camera(In any case, I don't see any handle() member function in the QVideoFrame class that would be used to get it if it was the case... despite the documentation mentioning it that in the detailed description)
What would be the proper way with Qt to get that memory mapped buffer available to openGL, other than doing a costly manual copy of the frame data using glTexImage2d ? (it's horrible, and I won't do that)
I assume that at this point the buffer in the QVideoFrame is still one of the "hardware buffer" obtained from Android / v4l2 / the driver, and not a copy by Qt already ?On Android (without Qt) if I'm not mistaken, the right way to do it would be to create a GL texture, use it with a SurfaceTexture which itself is linked to a Camera with Camera::setPreviewTexture. Then for every frame call updateTexImage() (of the SurfaceTexture), which internally uses glEGLImageTargetTexture2DOES to map the latest EGLImage buffer to our GL texture.
Access to the texture in the shader is done using GL_TEXTURE_EXTERNAL_OES as it's an external texture.Thanks for your help on this !
