Running QApplication::exec() from another thread (QCoreApplication, QGuiApplication)

gd2shoe

Otherwise known as the common question: "How do I run QApplication::exec() from a non-main thread?"

Preamble

I've found a lot of discussion about this online, but nowhere that all the relevant information is in one place. Maybe it is, and I missed it in my searching. I'm posting this here for two reasons. First, I want to share what I've found. Hopefully Google will index this page and it will be useful for others who follow. Second, I'm not an expert. If anyone wants to correct me, feel free. I'm happy to learn.

I'm using Qt For Python (PySide6), but this seems to be a common issue between mainline Qt and PySide. If I'm wrong, moderators are welcome to move this thread to the PySide forum.

I had my "Ah ha" moment while reading this thread thanks to @Chris-Kawa.

Currently, the formal documentation is incomplete, and somewhat inaccurate/misleading. Hopefully that will be rectified at some point.

Why this matters

Qt is principally a gui toolkit. It's other things too, but there is a general notion that if you're writing a Qt application, then the user interface is the primary thing that you care about. Everything else should go in a side thread somewhere -- that Qt deserves to own the one and only main() thread. That isn't always true.

There are many reasons why Qt might be employed to provide a gui in other contexts. The most obvious are threaded programs being ported from the command line to a windowed system. In such a program, it could require major refactoring and testing to move core control logic into a new thread. It makes more sense to manage QWigets/QApplication in the new thread instead.

It might be that relevant code or thread management isn't completely under your control or ownership. In my case, Jupyter doesn't want to part with the interactive python interpreter thread. Running QApplication.exec_() from one cell makes other cells wait until the event loop is closed. This partially defeats the purpose of wanting to use Jupyter to begin with (its extremely high interactivity).

But wait! Doesn't QApplication.exec() need to run in the main thread? Is there no alternative?

Main Thread

WARNING: QApplication was not created in the main() thread.

The first gotcha that crops up is the "main thread". The Qt "main thread" isn't actually the initial thread your process starts under. Instead, the Qt "main thread" is the first thread to create a QObject instance. So long as you create the QCoreApplication derivative prior to any other QObject, and run its event loop from the thread it's created in, it's happy.

It's important to note that QObjects, in general, are not threadsafe. There are some nifty signals/slots features that ameliorate this. But in general, anything to do with QWidgets must happen in the Qt "main thread" (where QApplication lives and works). Each QObject has a thread affinity and should only be referenced from within that thread. For more information on this, see the classic blog post You're doing it wrong..., or The Eight Rules of Multithreaded Qt, or any number of other wonderfully written treatises on Qt threading.

QCoreApplication Singleton

RuntimeError: Please destroy the QCoreApplication singleton before creating a new QCoreApplication instance.
RuntimeError: Please destroy the QGuiApplication singleton before creating a new QGuiApplication instance.
RuntimeError: Please destroy the QApplication singleton before creating a new QApplication instance.

Only one QCoreApplication instance can be created at a time (or it's derivatives). It should be created in the Qt "main thread" (which should also be its thread affinity). Its exec() function starts the main event loop, and should be run from the Qt "main thread".

The event loop will continue to run until QCoreApplication is asked to exit() or quit(). As near as I can tell, those aren't thread-safe! Maybe they are, but the documentation doesn't say so. Instead of calling these from another thread, it's probably much safer to post an event. postEvent() is explicitly thread-safe, and that will ensure that the handler running quit() or exit() will be doing so from the Qt "main thread".

QThread

QThread cannot be used to run the QApplication event loop. Why is that? QThread is a QObject. It is created before the thread that it manages is started. This means that the "main thread" gets defined before the QThread's run() function even begins. Creating QApplication and running exec() from QThread's run() inevitably results in QApplication complaining that it isn't in the main thread.

I don't know about the C++ implementation behavior, but in PySide6 under Windows, this somehow prevents the QApplication event loop from receiving event notifications from the underlying system (window, mouse, keyboard). Who knows what other problems this might also create?

Work Around

If you're tracking with me so far, the workaround is straightforward.

Don't put QApplication in a QThread.
Do initialize QApplication in its own thread, before creating any other QObjects. Find another way to manage this thread. Don't use QThread.
I'll say it again another way: If you do create other QObjects, be careful to avoid race conditions. Make sure the QApplication constructor finishes before any other QObject constructor begins. QThread is a QObject.
Do create all QWidget objects in the same thread as QApplication.
Do take care to run QApplication.quit() or QApplication.exit() in a thread-safe manner from the "main thread".
Do use QThread if you want to have additional non-gui Qt event loops. That's what QThread is intended for. Be careful about thread-safety pitfalls. Don't create QWidgets from a QThread.

Wrap-up

What did I miss? Did I get anything wrong? I'd be surprised if I didn't miss something important. Leave any thoughts or observations below. Maybe you'll help someone. (Maybe that someone will be me.)

Feel free to use either C++ or PySide as a context for discussion.

JKSH

@gd2shoe Thank you for this detailed, comprehensive, and correct post!

As far as I can see, everything you've written is correct. I can't see anything important that you've left out.

the Qt "main thread" is the first thread to create a QObject instance.

...

Only one QCoreApplication instance can be created at a time (or it's derivatives). It should be created in the Qt "main thread" (which should also be its thread affinity). Its exec() function starts the main event loop, and should be run from the Qt "main thread".

Bingo. These are the core points from which everything else derives from.

There is one more implication: If we want to destroy the QCoreApplication and create another one (without unloading the libraries from the process' memory), then we must keep the original "main thread" alive and create the new QCoreApplication in that same thread. Shutting down the original "main thread" means that we cannot create any more QCoreApplications in this process instance.

Currently, the formal documentation is incomplete, and somewhat inaccurate/misleading. Hopefully that will be rectified at some point.

Would you like to have a go at correcting it? (Both the documentation and the runtime warning, "WARNING: QApplication was not created in the main() thread.")