QWS VNC doesn't work on Portrait mode

[kumararajas]

Hi,

I am working on embedded product which has 7" LCD display. And the core is TI AM335x.

Since the display by default is, Landscape mode and I need to rotate the screen to portrait mode as per the requirements.

Since I need screen rotation, I use a QWS option "Transformed:Rot270" to rotate the screen. And it works cool.

Now I would like to extend the display to my linux as well. So, I use VNC for that.
When I tested VNC, it worked excellent in case of TI AM335x EVM board.
The reason is that, I have never rotated the screen in the EVM board.

Now, in my application, I need to rotate the screen.

The export option I have set is,
export QWS_DISPLAY="Multi: VNC:size=480x640:0 Transformed:Rot270:LinuxFb:size=480x640:1 Transformed:Rot270 :2"

With this In the desktop, I could see the VNC viewer screen was rotated and displayed as I wanted. And also, I have used the mouse to operate the UI and it was working perfect.

But in the LCD UI, I could see two UI's of actual UI. Meaning, One in the landscape mode and other one in the portrait mode.
And I can actually operate the one which is in portrait mode. The landscape mode UI was actually from VNC (may be).

Then later, I was trying various combinations for QWS, but could not corner the problem.

Then I googled to see if there are fixes. That time, I came across this:
https://bugreports.qt.io/browse/QTBUG-12290
It looks like it is a bug from Qt related to screen rotation.

I wanted to apply this patch and see if it solves my problem.

After applying the patch, my observations are

• Both LCD and Linux shows the UI exactly same.
• When I operate the UI from Linux, it works perfect.
• When I operate the UI from LCD, it doesnt work. It looks like the touch inputs are still without screen rotate.
  Visualize the screen in portrait mode (rotated screen). Meaning, assume there is a button in top left corner. If I click the button from Linux VNC, yes it works. When I touch the same position - top left in the LCD UI, there is no response. But, if I touch the bottom left, button responds. It seems like the touch inputs are not rotated.

I tried my best to explain the problem. Please let me know if you have any thoughts on this.

Thank you,
Kumara

[SGaist]

Hi,

I'd look at the sendMouseEvent patch part, it's probably there that the dispatching is not done correctly

[kumararajas]

@SGaist Hi Sam, Thank you for taking time and looking at the implementation.

How to go about?

Thanks,
Kumara

[kumararajas]

@SGaist This was the else part added as part of the patch (that is attached in Qt bug)

    if (qt_screen->isTransformed()) 
    {
        QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
        tpos = qt_screen->mapFromDevice(pos, s);
    } 
    else 
    {
        //tpos = pos;
        // Only in case of Multi Display
        // If main screen is not transformed, then check if sub screens are transformed
        QList<QScreen*> screens = qt_screen->subScreens();
        if (screens.size() && !bRemote) 
        {
            for (int i = 0; i < screens.size(); ++i) 
            {
                QScreen *screen = screens.at(i);
                if (screen->isTransformed())
                {
                    QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                    tpos = screen->mapFromDevice(pos, s);
                    break;
                }
            }
        }
    }

[kumararajas]

@SGaist This Qt bug is not fixed in any of Qt 4.8 versions?

[SGaist]

It's still the same in 4.8.7

[kumararajas]

@SGaist Ok Sam. Do you see any way to fix this problem?

Thank you.

[SGaist]

If I understand the patch correctly, it only check other screens if the main screen is not transformed. Since you have more that one screen transformed, you should check all screens (main + sub-screens)

[kumararajas]

@SGaist Then it means that, the else part of code, should be done for if condition as well?

    if (qt_screen->isTransformed()) 
    {
        QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
        tpos = qt_screen->mapFromDevice(pos, s);
    } 
    //tpos = pos;
    // Only in case of Multi Display
    // If main screen is not transformed, then check if sub screens are transformed
    QList<QScreen*> screens = qt_screen->subScreens();
    if (screens.size() && !bRemote) 
    {
        for (int i = 0; i < screens.size(); ++i) 
        {
            QScreen *screen = screens.at(i);
            if (screen->isTransformed())
            {
                QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                tpos = screen->mapFromDevice(pos, s);
                break;
            }
        }
    }

Is it like that?

[SGaist]

I'd remove also the bRemote, just check them all

[kumararajas]

@SGaist Hmm, let me test this.

Just for better understanding attaching the modified code:

if (qt_screen->isTransformed()) 
    {
        QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
        tpos = qt_screen->mapFromDevice(pos, s);
    } 
    //tpos = pos;
    // Only in case of Multi Display
    // If main screen is not transformed, then check if sub screens are transformed
    QList<QScreen*> screens = qt_screen->subScreens();
    if (screens.size()) 
    {
        for (int i = 0; i < screens.size(); ++i) 
        {
            QScreen *screen = screens.at(i);
            if (screen->isTransformed())
            {
                QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                tpos = screen->mapFromDevice(pos, s);
                break;
            }
        }
    } 

I'll test and keep you posted..

[kumararajas]

No success yet!

While running the application with Multi for VNC, still the touch inputs on the device is still not rotated.

[SGaist]

Silly idea (and just to verify) but what about just dropping the if and do the transform for all screens anyway ?

[kumararajas]

Just for more information:

export QWS_DISPLAY="Multi: VNC:0:size=480x640 Transformed:rot0"

This displays the UI in a landscape mode and the touch input works as per the UI.

export QWS_DISPLAY="Multi: VNC:0:size=480x640 Transformed:rot90"

This rotates the UI to 90 degrees. But the touch input is as same as 0 degrees.

This clearly indicates that the display transformation works good but the transformation to touch input mapping is not working.

[kumararajas]

@SGaist Sam, can you please send me the code snippet to be on same page?

Thanks,
Kumara

[kumararajas]

@SGaist You mean to try this kind of code:

QList<QScreen*> screens = qt_screen->subScreens();
if (screens.size()) 
{
    for (int i = 0; i < screens.size(); ++i) 
    {
        QScreen *screen = screens.at(i);
        QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
        tpos = screen->mapFromDevice(pos, s);
        break;
    }
}

instead of

if (qt_screen->isTransformed()) 
    {
        QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
        tpos = qt_screen->mapFromDevice(pos, s);
    } 
    else 
    {
        //tpos = pos;
        // Only in case of Multi Display
        // If main screen is not transformed, then check if sub screens are transformed
        QList<QScreen*> screens = qt_screen->subScreens();
        if (screens.size() && !bRemote) 
        {
            for (int i = 0; i < screens.size(); ++i) 
            {
                QScreen *screen = screens.at(i);
                if (screen->isTransformed())
                {
                    QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                    tpos = screen->mapFromDevice(pos, s);
                    break;
                }
            }
        }
    }

Sam, if you confirm, I can make these changes and build my packages.

Thanks,
Kumara

[SGaist]

Yes that's what I had in mind, without forgetting the main screen first.

[kumararajas]

@SGaist Oh ya, I missed the main screen

So here is the code which I can do the test:

// Main screen
QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
tpos = qt_screen->mapFromDevice(pos, s);

// Subscreens
QList<QScreen*> screens = qt_screen->subScreens();
if (screens.size()) 
{
    for (int i = 0; i < screens.size(); ++i) 
    {
        QScreen *screen = screens.at(i);
        QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
        tpos = screen->mapFromDevice(pos, s);
        break;
    }
}

Right?

[kumararajas]

@SGaist Below the patch code, I see

    QPoint tpos;
    // Main screen
    QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
    tpos = qt_screen->mapFromDevice(pos, s);

    // Sub screens
    QList<QScreen*> screens = qt_screen->subScreens();
    if (screens.size()) 
    {
        for (int i = 0; i < screens.size(); ++i) 
        {
            QScreen *screen = screens.at(i);
            QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
            tpos = screen->mapFromDevice(pos, s);
            break;
        }
    }

    if (qt_last_x) {
         *qt_last_x = tpos.x();
         *qt_last_y = tpos.y();
    }
    QWSServer::mousePosition = tpos;
    qwsServerPrivate->mouseState = state;

We use the tpos and mouse the mousePosition just once. Even though, we have many subscreens, we just do it once.

Is this correct?

Should we not do this in a loop for all the screens?

Please let me know.

[kumararajas]

Looks like I am in a wrong direction.

I suspect "Transformed" driver is not having problem.

I think, it is "multi" screen driver is what not working good.

Because, when VNC option is selected, Transformed works food.
When LinuxFb is selected, Transformed works good.

When both are used, there lies the problem.

I guess, we may have to look at Multi Screen driver source code to see how it has been implemented.

Can you please point me to the right location?

Thanks,
Kumara