TUIO multi-touch

3n3l

Hello,

I'm building an application with Qt Quick that needs to be controlled by a TUIO-based touch surface. Apparently Qt has native TUIO multi-touch support, you can even see it in the documentation, but I couldn't find anything in the documentation regarding the reception of the TUIO messages (which are wrapped in UDP packets).

Does anyone have any idea on how to connect the sending process to my Qt application without building my own UDP socket and making everything too complicated?
Is there a native solution for receiving this TUIO messages that I just can't find in the documentation?

jeremy_k

There's some documentation in the plugin readme.
https://code.qt.io/cgit/qt/qtbase.git/tree/src/plugins/generic/tuiotouch/README.md:

By default, you must direct TUIO packets to the IP of the machine the application
is running on, protocol UDP, port 3333. If you want to customize the port, you
may provide a port number like this:

qmlscene foo.qml -plugin TuioTouch:udp=3333

3n3l

Thank you! But looking at the TuioTouch plugin it seems to be loaded with QT_QPA_GENERIC_PLUGINS which only generates QTabletEvents.
What I'm trying to achieve is to use TouchPoints (and MultiPointTouchAreas). These QTabletEvents might not even exist for Qt Quick?!

jeremy_k

What version of Qt is this targeting?

I've looked at Qt 5, 6, and the linked documentation above, and see no mention of QTabletEvent in connection with the TuioTouch plugin. The plugin works with touch events, which under some circumstances can be exposed as Quick Touch objects.

Rather than referring to a decade-old repository, try https://code.qt.io/cgit/qt/qtbase.git/tree/src/plugins/generic/tuiotouch for more relevant code (unless you're stuck with decade-old Qt).

3n3l

You're right about the documentation, the part of it I was referring mentions another plugin. But I did test this and it did not generate any touch events, or atleast my MultiPointTouchArea wasn't receiving any, after that I found the linked documentation and thought that this just isn't working.

I'm targeting Qt 6.7.

How would one expose these touch events as Quick Touch objects? This could be the solution I'm looking for!

jeremy_k

@3n3l said in TUIO multi-touch:

it did not generate any touch events, or atleast my MultiPointTouchArea wasn't receiving any

Try using an event filter on the QCoreApplication to determine if related events are being delivered to something. If not, the TUIO plugin might be to blame. I would look for the expected network traffic. If events are delivered, follow the chain of event delivery attempts. Is it possible that the intended receiver has the enabled property set to false, or acceptsTouchEvents() == false?

3n3l

Minimalistic example test.qml that doesn't work when being called with qml test.qml -plugin TuioTouch:

import QtQuick 2.10
import QtQuick.Window 2.15
import QtQuick.Controls 2.10


Window {
    width: 1920
    height: 1080
    color: "black"
    visible: true

    MultiPointTouchArea {
        anchors.fill: parent
        onPressed: (touchPoints) => {
            console.log("PRESSED!", touchPoints);
        }
    }
}

The plugin is definitely loaded, I'm receiving error messages from it when sent messages contain time tags, so the basic functionality should work, sending other types of messages produces no output at all.

I'm not even sure how to load a plugin if I'm loading the qml application from PySide (or C++ for that matter, I'm guessing one can use CMake there to load the plugin? But I'm using PySide anyway for this application), can't find any documentation for this either, I'm gonna try out the event filter after I've found a solution for this.

3n3l

Installing an event filter on QGuiApplication receives only a QtCore.QEvent(QEvent::SockAct) for every tuio message.

Looking at the TuioTouch plugin it seems to create a QWindowSystemInterface::TouchPoint for every message, but these are not the Qml TouchPoints, are they?

jeremy_k

@3n3l said in TUIO multi-touch:

Installing an event filter on QGuiApplication receives only a QtCore.QEvent(QEvent::SockAct) for every tuio message.

Looking at the TuioTouch plugin it seems to create a QWindowSystemInterface::TouchPoint for every message, but these are not the Qml TouchPoints, are they?

QWindowSystemInterface::TouchPoint is an intermediate interface. When the input plugin delivers a touch event, it doesn't know that the eventual recipient will be a Quick item. If QWSI::TouchPoint instances being created, look at their coordinates, and the window, Item, etc that the application expects to receive touch input. Sometimes there is a mismatch, such as a rotation by 90 or 180 degrees.

3n3l

Got it working, the issue was that my sending application would be in focus and then my qml application would not receive touch events when being out of focus (What?).
I read the source code of the plugin quite a few times and completely missed this part:

static bool forceDelivery = qEnvironmentVariableIsSet("QT_TUIOTOUCH_DELIVER_WITHOUT_FOCUS");

-> this environment variable needs to be set to get it working (export QT_TUIOTOUCH_DELIVER_WITHOUT_FOCUS=1).

@jeremy_k thank you very much for your help!

jeremy_k

@3n3l said in TUIO multi-touch:

Got it working, the issue was that my sending application would be in focus and then my qml application would not receive touch events when being out of focus (What?).

I've been bitten by that issue category. Using a virtual machine that won't know it has lost focus from the physical input hardware is another solution, although not as convenient as the environment variable.