Dump QImage raw pixel data into `std::vector<char>`

[Christian Ehrlicher]

@abmyii malloc allocates memory - nothing more.
The asan output is fine - no problems in your code. Only somwhere memory is allocated with new and deallocated with free() or allocated with malloc and deallocated with delete which is not allowed.

[abmyii]

@Christian-Ehrlicher That's good to hear! I wonder what's causing the issue, then - as soon as I remove -fsanitize=address it starts failing again. Can I leave -fsanitize=address in for a release binary? Even if it was OK, I'd need to find a way to fix the above issue when running without the env variable. I'll have a look to see if I can find the offending lines of code.

Edit: Didn't find any (especially with my untrained eyes...). The only malloc is the image one.

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

it starts failing again.

This really sounds more like a threading issue.

[abmyii]

@Christian-Ehrlicher How does having -fsantize=address cause it to stop failing? Also is there any way to debug it?

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

How does having -fsantize=address cause it to stop failing?

When it's a thread issue then the timing is now different

Also is there any way to debug it?

With a debugger, yes. Examine your threads when it crashes

[abmyii]

@Christian-Ehrlicher

(gdb) thread apply all bt

Thread 6 (Thread 0xb0dfdbc0 (LWP 4458)):
#0  0xb6e433a2 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
#1  0x00000c00 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 5 (Thread 0xb15fdbc0 (LWP 4457)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e282c6 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 4 (Thread 0xb20b0bc0 (LWP 4455)):
#0  0xb69634e2 in select () at ../sysdeps/unix/syscall-template.S:84
#1  0xb6e28868 in ?? () from /usr/lib/arm-linux-gnueabihf/libvncserver.so.1
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 3 (Thread 0xb2f0abc0 (LWP 4454)):
#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:46
#1  0xb68badba in __pthread_cond_wait (cond=0x72f58, mutex=0x728f0) at pthread_cond_wait.c:186
#2  0xb2f93ff4 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 2 (Thread 0xb3dd9bc0 (LWP 4453)):
#0  0xb6961c00 in poll () at ../sysdeps/unix/syscall-template.S:84
#1  0xb684c7c8 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#2  0xb6851038 in ?? () from /usr/lib/arm-linux-gnueabihf/libmircommon.so.7
#3  0xb6d9edc8 in ?? () from /usr/lib/arm-linux-gnueabihf/libstdc++.so.6
#4  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#5  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

Thread 1 (Thread 0xb40ea000 (LWP 4441)):
#0  0xb69658d6 in munmap () at ../sysdeps/unix/syscall-template.S:84
#1  0xb692480e in munmap_chunk (p=<optimized out>) at malloc.c:2860
#2  0xb6ae6f12 in QImageData::~QImageData() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#3  0xb6ae7122 in QImage::~QImage() () from /usr/lib/arm-linux-gnueabihf/libQt5Gui.so.5
#4  0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()
#5  0x0001b130 in (anonymous namespace)::do_screencast((anonymous namespace)::EGLSetup const&, mir::geometry::Size const&, int, double, std::ostream&) ()
#6  0x0001c446 in main ()

Quite indecipherable to me but the last (#1) thread's backtrace looks promising...

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

#4 0x00019df6 in (anonymous namespace)::read_pixels(int, unsigned int, mir::geometry::Size const&, std::vector<char, std::allocator<char> >&) ()

What happens here?
Also try to install the debug symbols of libvncserver

[abmyii]

@Christian-Ehrlicher

void read_pixels(int bpp, GLenum format, mir::geometry::Size const& size, std::vector<char> &fb)
{
    auto width = size.width.as_uint32_t();
    auto height = size.height.as_uint32_t();

    // Read pixels into image (https://community.khronos.org/t/render-to-texture-under-qt/68430/3)
    QImage image(width, height, QImage::Format_RGBA8888);
    glReadPixels(0, 0, width, height, format, GL_UNSIGNED_BYTE, image.bits());

    if (mirror)
	image = image.mirrored(true, true);

    if (landscape) {
        //QPoint center = image.rect().center();
        QMatrix matrix;
        matrix.translate(0, 0);
        matrix.rotate(90);

        image = image.transformed(matrix);
    }

    std::cout << "FB size: " << fb.size() << ", Image byteCount: " << image.byteCount() << std::endl;
    std::memcpy(fb.data(), (const char *)image.constBits(), image.byteCount());
#ifdef DEBUG
    unsigned int sum = 0;
    unsigned int *pui = (unsigned int *) buffer;
    for (int i = 0; i < width * height; i++) {
        sum += *(pui + i);
    }
    printf("read pixels sum 0x%x\n", sum);
#endif
}

Sure, I'll try that.

[abmyii]

@Christian-Ehrlicher

Thread 6 "mirvncserver" received signal SIGBUS, Bus error.
[Switching to Thread 0xb0db8bc0 (LWP 3495)]
0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
611	tight.c: No such file or directory.

(gdb) thread apply all bt

Thread 6 (Thread 0xb0db8bc0 (LWP 3495)):
#0  0xb6e433b2 in CheckSolidTile32 (cl=0xb1700470, needSameColor=0 '\000', colorPtr=0xb0db8438, h=16, w=16, y=-1231936575, x=20) at tight.c:611
#1  CheckSolidTile (cl=cl@entry=0xb1700470, x=x@entry=0, y=y@entry=465, w=w@entry=16, h=h@entry=16, colorPtr=colorPtr@entry=0xb0db8438, 
    needSameColor=needSameColor@entry=0 '\000') at tight.c:570
#2  0xb6e45aae in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=x@entry=0, y=433, w=w@entry=432, h=h@entry=335) at tight.c:388
#3  0xb6e45fba in SendRectEncodingTight (cl=cl@entry=0xb1700470, x=0, y=0, w=432, h=768) at tight.c:450
#4  0xb6e460f6 in rfbSendRectEncodingTight (cl=cl@entry=0xb1700470, x=<optimized out>, y=<optimized out>, w=<optimized out>, h=h@entry=768) at tight.c:275
#5  0xb6e2cf7c in rfbSendFramebufferUpdate (cl=cl@entry=0xb1700470, givenUpdateRegion=givenUpdateRegion@entry=0xb0200470) at rfbserver.c:3165
#6  0xb6e27e0c in clientOutput (data=0xb1700470) at main.c:497
#7  0xb68b65b4 in start_thread (arg=0x0) at pthread_create.c:335
#8  0xb6967c5c in ?? () at ../sysdeps/unix/sysv/linux/arm/clone.S:89 from /lib/arm-linux-gnueabihf/libc.so.6
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

[Christian Ehrlicher]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

std::vector<char> &fb

Is this used in another thread?

Otherwise I don't see anything obvious.

[abmyii]

@Christian-Ehrlicher I think the while loop is in another thread and VNC is accessing fb.data() throughout execution.

[Christian Ehrlicher]

@abmyii So you have your answer. Concurrent access to a memory region from two different threads.

[abmyii]

@Christian-Ehrlicher Ah I see, that makes perfect sense now! Is there a simple solution to this? I tried the following which didn't work:

std::vector<char> temp_fb(frame_size_bytes, 0);
std::memcpy(temp_fb.data(), (const char *)image.constBits(), image.byteCount());
fb = temp_fb

[Christian Ehrlicher]

Use mutexes or copy the data - I would suggest reading a little bit on how threading works. E.g. https://doc.qt.io/qt-5/threads-synchronizing.html - c++ uses the same mechanisms

[abmyii]

Whoops, I've been saying malloc (which is apparently thread-safe) all along when in fact the actual function I'm using is memcpy... Does that change things and/or can I use malloc instead?

[JonB]

@abmyii
Hence what @Christian-Ehrlicher has been saying above. malloc and memcpy do totally different things, it makes no sense to ask whether one can be used in place of the other....

[abmyii]

@JonB I understand - I got confused due to my unfamiliarity with the functions but I never asked if I could use malloc instead of memcpy - I asked if I could use push_back instead of malloc (sic).

[JonB]

@abmyii said in Dump QImage raw pixel data into &#x60;std::vector<char>&#x60;:

I never asked if I could use malloc instead of memcpy [sic.]

in fact the actual function I'm using is memcpy... Does that change things and/or can I use malloc instead? [*sic.]

If you say so! I'll leave others to untangle, who clearly understand what you are saying better than I :) Don't worry about explaining to me.

[abmyii]

@JonB OK, you got me! Too much to keep track of...

I tried this and it worked, but is quite a lot slower than memcpy:

std::vector<unsigned char> vectorBuffer(charBuffer, charBuffer + length);
fb = vectorBuffer;

[JonB]

@abmyii
I am lost in the complexities of your questions :) But I would suggest heeding @Christian-Ehrlicher's advice, he is usually right! He last offered:

Use mutexes or copy the data

So why not pursue that?