How to Properly Set a QCursor with a Mask？

Gao.xiangyang

I am implementing a remote desktop protocol and need to display the cursor bitmap from the remote desktop on the local client. This involves setting mouse pointer types with monochrome bitmaps. Additionally, I require cross-platform functionality, which is why I am using QCursor in Qt.

On windows platform.
I have now learned how to set a cursor with a mask using the Win32 API.And I have achieved the desired result.

Due to the lengthy binary content, I haven't posted the code here. For reference, you can see this example: https://learn.microsoft.com/en-us/windows/win32/menurc/using-cursors

HINSTANCE inst;
HCURSOR cursor = CreateCursor(inst,
                              pCursorData->xHotspot,
                              pCursorData->yHotspot,
                              pCursorData->width,
                              pCursorData->height,
                              (void*)pCursorData->pixelData,
                              (void*)(pCursorData->pixelData + size));

SetCursor(cursor);

Afterward, I utilized QImage to convert my monochrome bitmap and proceeded to configure QCursor. However, the result I obtained differed from what is typically achieved when employing the native WIN32 API interface.

// `height` is 2 bitmap;
int size =  pCursorData->width * pCursorData->height / 2 / 8;
QImage     image(pCursorData->pixelData,          pCursorData->width,pCursorData->height / 2,QImage::Format_Mono);
QImage maskImage((pCursorData->pixelData + size), pCursorData->width,pCursorData->height / 2,QImage::Format_Mono);

pixmap = QPixmap::fromImage(image);
pixmap.setMask(maskBitmap);
QCursor cursor(pixmap, pCursorData->xHotspot, pCursorData->yHotspot);

renderWidget->setCursor(cursor);

Ultimately, after consulting additional documentation, I revised the code accordingly.

/**
 * type: MONOCHAROME has (AND bitmap) and (XOR bitmap).
 *  1 bit = 1 pixed
 *
 * Windows:
 * AND bitmap  XOR bitmap  Display
 *   0          0           Black
 *   0          1           White
 *   1          0           Screen
 *   1          1           Reverse screen
 *
 * QCursor:
 * The cursor bitmap (B) and mask (M) bits are combined like this:
 *  B=1 and M=1 gives black.
 *  B=0 and M=1 gives white.
 *  B=0 and M=0 gives transparent.
 *  B=1 and M=0 gives an XOR'd result under Windows, undefined results on all other platforms.
 * Use the global Qt color Qt::color0 to draw 0-pixels and Qt::color1 to draw 1-pixels in the bitmaps.
 */

int pitch = pCursorData->dataLen / pCursorData->height;
bool andPixel;
bool xorPixel;
// `height` is 2 bitmap;
for(int y=0; y < pCursorData->height / 2; y++)
{
    for(int x=0; x < pCursorData->width; x++)
    {
        quint8 bitMask = 0x80 >> (x % 8);
        quint8 uint8_x = x / 8; // width is bit size

        andPixel = pCursorData->pixelData[y * pitch + uint8_x] & bitMask;
        xorPixel = pCursorData->pixelData[(y + pCursorData->height / 2) * pitch + uint8_x] & bitMask;

        if (!andPixel && !xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color1));
            maskImage.setPixelColor(x, y, QColor(Qt::color1));
        } else if (!andPixel && xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color0));
            maskImage.setPixelColor(x, y, QColor(Qt::color1));
        } else if (andPixel && !xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color0));
            maskImage.setPixelColor(x, y, QColor(Qt::color0));
        } else if (andPixel && xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color1));
            maskImage.setPixelColor(x, y, QColor(Qt::color0));
        }
    }
}

pixmap = QPixmap::fromImage(image);
pixmap.setMask(maskBitmap);
QCursor cursor(pixmap, pCursorData->xHotspot, pCursorData->yHotspot);

renderWidget->setCursor(cursor);

Despite all my efforts, they proved to be in vain. I later delved into the Qt source code to understand the implementation of QCursor and discovered it involved numerous bitwise operations, which were quite challenging for me to grasp in a short period. Therefore, I am reaching out to the experts in this field for guidance: How should one convert Windows' AND mask and XOR mask bitmaps into a QCursor?

Gao.xiangyang

I have made continuous attempts and discovered some patterns in color conversion. The official documentation of QCursor mentions Qt::color0 and Qt::color1, which indeed follow certain rules. However, I found that the results I obtained are the opposite of the bit values.

As mentioned in my comments, I am currently inverting the bit values of the Windows original AND mask and XOR mask to match Qt::color0 and Qt::color1 as mentioned in the QCursor documentation.

        /**
         * type: MONOCHAROME has (AND bitmap) and (XOR bitmap).
         *  1 bit = 1 pixed
         *
         * Windows:
         * AND bitmap  XOR bitmap  Display
         *   0          0           Black
         *   0          1           White
         *   1          0           Screen
         *   1          1           Reverse screen
         *
         * QCursor:
         * The cursor bitmap (B) and mask (M) bits are combined like this:
         *  B=1 and M=1 gives black.
         *  B=0 and M=1 gives white.
         *  B=0 and M=0 gives transparent.
         *  B=1 and M=0 gives an XOR'd result under Windows, undefined results on all other platforms.
         * Use the global Qt color Qt::color0 to draw 0-pixels and Qt::color1 to draw 1-pixels in the bitmaps.
         *
         * Actual results:
         *   0          0           Black
         *   1          0           White
         *   1          1           transparent screen
         *   0          1           Reverse screen
         */


        int size =  pCursorData->width * pCursorData->height / 2 / 8;
        if(pCursorData->dataLen / 2 != size)
        {
            qWarning() << "[CMouseModule::SetServerCursor] set Cursor Data dataLen failed.";
            return;
        }

        QImage     image(pCursorData->width, pCursorData->height / 2, QImage::Format_Mono);
        QImage maskImage(pCursorData->width, pCursorData->height / 2, QImage::Format_Mono);

        if(image.sizeInBytes() != size)
        {
            qWarning() << "[CMouseModule::SetServerCursor] bitmap size not eq Cursor Data image size.";
            return;
        }

        uchar* imageBits = image.bits();
        uchar* maskImageBits = maskImage.bits();

        int pitch = pCursorData->dataLen / pCursorData->height;
        bool andPixel;
        bool xorPixel;
//        // `height` is 2 bitmap;
        for(int y=0; y < pCursorData->height / 2; y++)
        {
            for(int x=0; x < pCursorData->width; x++)
            {
                quint8 bitMask = 0x80 >> (x % 8);
                quint8 uint8_x = x / 8; // width is bit size

                andPixel = pCursorData->pixelData[y * pitch + uint8_x] & bitMask;
                xorPixel = pCursorData->pixelData[(y + pCursorData->height / 2) * pitch + uint8_x] & bitMask;

                if(!andPixel && !xorPixel)
                {
                    imageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                    maskImageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                } else if(!andPixel && xorPixel)
                {
                    imageBits[y * pitch + uint8_x] |= bitMask; // 1
                    maskImageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                } else if(andPixel && !xorPixel)
                {
                    imageBits[y * pitch + uint8_x] |= bitMask; // 1
                    maskImageBits[y * pitch + uint8_x] |= bitMask; // 1
                } else if(andPixel && xorPixel)
                {
                    imageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                    maskImageBits[y * pitch + uint8_x] |= bitMask; // 1
                }
            }
        }

        QBitmap bitmap;
        QBitmap maskBitmap;
        bitmap = QBitmap::fromImage(image);
        maskBitmap = QBitmap::fromImage(maskImage);
        QCursor cursor(bitmap, maskBitmap, pCursorData->xHotspot, pCursorData->yHotspot);
renderWidget->setCursor(cursor);

Pl45m4

@Gao-xiangyang said in How to Properly Set a QCursor with a Mask？:

How should one convert Windows' AND mask and XOR mask bitmaps into a QCursor?

I have never done this, but there is

and

which convert HBITMAP and HICON (= HCURSOR) to and from QImage/QPixmap.
(every QImage can be converted to QPixmap and vice versa)

But you need Qt6 for this.

In Qt5 or below these functions are in QtWin namespace

https://doc.qt.io/qt-5/qtwin.html

which is obsolete and some parts already got removed in Qt6

Gao.xiangyang

@Pl45m4
Thank you for your response.
I was able to find the code related to winextras using the information you provided. I found Q_GUI_EXPORT QPixmap qt_pixmapFromWinHICON(HICON icon).
But it is too complicated for me because the AND mask and XOR mask data I obtained are fixed, and I need to generate a QCursor on Linux, macOS, and Windows platforms.

I need the simplest method to convert the "AND mask and XOR mask -->> QCursor".

Gao.xiangyang

I have made continuous attempts and discovered some patterns in color conversion. The official documentation of QCursor mentions Qt::color0 and Qt::color1, which indeed follow certain rules. However, I found that the results I obtained are the opposite of the bit values.

As mentioned in my comments, I am currently inverting the bit values of the Windows original AND mask and XOR mask to match Qt::color0 and Qt::color1 as mentioned in the QCursor documentation.

        /**
         * type: MONOCHAROME has (AND bitmap) and (XOR bitmap).
         *  1 bit = 1 pixed
         *
         * Windows:
         * AND bitmap  XOR bitmap  Display
         *   0          0           Black
         *   0          1           White
         *   1          0           Screen
         *   1          1           Reverse screen
         *
         * QCursor:
         * The cursor bitmap (B) and mask (M) bits are combined like this:
         *  B=1 and M=1 gives black.
         *  B=0 and M=1 gives white.
         *  B=0 and M=0 gives transparent.
         *  B=1 and M=0 gives an XOR'd result under Windows, undefined results on all other platforms.
         * Use the global Qt color Qt::color0 to draw 0-pixels and Qt::color1 to draw 1-pixels in the bitmaps.
         *
         * Actual results:
         *   0          0           Black
         *   1          0           White
         *   1          1           transparent screen
         *   0          1           Reverse screen
         */


        int size =  pCursorData->width * pCursorData->height / 2 / 8;
        if(pCursorData->dataLen / 2 != size)
        {
            qWarning() << "[CMouseModule::SetServerCursor] set Cursor Data dataLen failed.";
            return;
        }

        QImage     image(pCursorData->width, pCursorData->height / 2, QImage::Format_Mono);
        QImage maskImage(pCursorData->width, pCursorData->height / 2, QImage::Format_Mono);

        if(image.sizeInBytes() != size)
        {
            qWarning() << "[CMouseModule::SetServerCursor] bitmap size not eq Cursor Data image size.";
            return;
        }

        uchar* imageBits = image.bits();
        uchar* maskImageBits = maskImage.bits();

        int pitch = pCursorData->dataLen / pCursorData->height;
        bool andPixel;
        bool xorPixel;
//        // `height` is 2 bitmap;
        for(int y=0; y < pCursorData->height / 2; y++)
        {
            for(int x=0; x < pCursorData->width; x++)
            {
                quint8 bitMask = 0x80 >> (x % 8);
                quint8 uint8_x = x / 8; // width is bit size

                andPixel = pCursorData->pixelData[y * pitch + uint8_x] & bitMask;
                xorPixel = pCursorData->pixelData[(y + pCursorData->height / 2) * pitch + uint8_x] & bitMask;

                if(!andPixel && !xorPixel)
                {
                    imageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                    maskImageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                } else if(!andPixel && xorPixel)
                {
                    imageBits[y * pitch + uint8_x] |= bitMask; // 1
                    maskImageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                } else if(andPixel && !xorPixel)
                {
                    imageBits[y * pitch + uint8_x] |= bitMask; // 1
                    maskImageBits[y * pitch + uint8_x] |= bitMask; // 1
                } else if(andPixel && xorPixel)
                {
                    imageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                    maskImageBits[y * pitch + uint8_x] |= bitMask; // 1
                }
            }
        }

        QBitmap bitmap;
        QBitmap maskBitmap;
        bitmap = QBitmap::fromImage(image);
        maskBitmap = QBitmap::fromImage(maskImage);
        QCursor cursor(bitmap, maskBitmap, pCursorData->xHotspot, pCursorData->yHotspot);
renderWidget->setCursor(cursor);