How to use Qt for a GUI inside an existing game engine?
-
Hi,
We want to use Qt to generate a futuristic user interface inside a game (engine). We are puzzling how to do this for a long time, but still do not know what is the way to go. Because it differs so much from what normally is the case.
Note that we do not want to use Qt as the game engine. We have our own game engine. Inside this game engine we have some vehicle that should have a futuristic user interface. We would like to create this user interface using Qt. But how?What we would like to do:
- use Qt to create and operate a modern looking user interface with lots of animations.
- use input from our game engine for the actual mouse position and button states for Qt.
- we would like to synchronize the frame drawing of this interface to our game engine (e.g. call a function to generate the next frame?)
- and we would like to copy the generated gui surface each frame to some texture inside our own game engine.
Questions we think need to be answerred:
- Can we have control over Qt frame generation? Like calling a function to generate the next frame.
- Can we overrule mouse clicks with input from our game engine (telling Qt the effective mouse position and button states)?
- Can we somehow copy the contents of the generated Qt frame or window to a know picture format (like .png)?
- Would it be better to use Widgets or QML?
Or should we do it completely different?
Any advice is welcome because we know to little about the internals of Qt.
-
I guess that for the specific way you want to do this there is no good answer right away. I am not sure if anyone has ever tried this. Here are a few thoughts on this:
First of all, I expect you need to run Qt in a separate thread. Qt needs the event loop to work at all. Since you do not want for Qt to react to mouse events itself I would say you need to go against conventional wisdom an not run Qt in the main thread. Otherwise it would get all mouse input directly and you would have to forward mouse events to your game engine. So, if started in a new thread, I would expect it does not get any mouse input from the operating system. You can then use
QMetaObject::invokeMethod(...), e.g. with QMousePressEvent, to put mouse events into the event loop of Qt.In general, Qt can draw to a QPixmap which would probably help a lot in your case. I haven't come up with an idea so far how to actually make the general paint routine paint to the pixmap. I am more knowledgeable with QWidgets, but not with QML at all. With QWidget if you are doing the drawing by hand, you could have a QPainter on the QPixmap which would be passed as parameter to QWidget::paint(). Also, for QWidget there are draw routines in the background to draw the specific elements like checkboxes, buttons, labels, etc. However, since you mentioned you want to have a futuristic user interface (I expect a lot of styling), it seems that all this does not help a lot. In general, I would expect QML to be a better fit to have a fancy user interface. Another thought (though likely a lot more involved) is that Qt has different backends for drawing, e.g. for Window, X Window, macOS, iOS, ... There is also a backend for embedded hardware that does expect a window manager to be running. This last backend could be a basis to write your own backend which draws to a QPixmap (offscreen) which is easily accessible in your game engine.
I don't fully get why you would want to use Qt in this context at all. What you describe will force Qt to work in a way it is not built for. You will always have to fight the normal way Qt usually behaves. Your approach certainly is not supported by Qt at all. Concerning your synchronization of frame drawing sounds a lot like immediate mode UIs. So, why don't you just use IMGUI? BTW, for a futuristic UI you need to do a lot of styling. It would be much easier to just do all the drawing yourself. You say you want to manage button states and mouse positions yourself. I expect you don't want to use any layouts (for automatic resizing). This means that the only thing you are missing in your description is a "collision detection" of your mouse pointer with the buttons. I bet you can easily implement this yourself (or reuse some existing collision detection). Then you have your own fully functioning UI.
The only way Qt would make sense in the context of a game would be to use Qt as usual. There is a way to combine QWidgets with OpenGL which is native to the Qt framework. This would mean your game would have to render to Qt's OpenGL widget. It seems this is not what you want.
-
Hi,
Is it a custom engine ?
Qt can be used for example with Ogre3d.You might find there something to help you get started.
-
I guess that for the specific way you want to do this there is no good answer right away. I am not sure if anyone has ever tried this. Here are a few thoughts on this:
First of all, I expect you need to run Qt in a separate thread. Qt needs the event loop to work at all. Since you do not want for Qt to react to mouse events itself I would say you need to go against conventional wisdom an not run Qt in the main thread. Otherwise it would get all mouse input directly and you would have to forward mouse events to your game engine. So, if started in a new thread, I would expect it does not get any mouse input from the operating system. You can then use
QMetaObject::invokeMethod(...), e.g. with QMousePressEvent, to put mouse events into the event loop of Qt.In general, Qt can draw to a QPixmap which would probably help a lot in your case. I haven't come up with an idea so far how to actually make the general paint routine paint to the pixmap. I am more knowledgeable with QWidgets, but not with QML at all. With QWidget if you are doing the drawing by hand, you could have a QPainter on the QPixmap which would be passed as parameter to QWidget::paint(). Also, for QWidget there are draw routines in the background to draw the specific elements like checkboxes, buttons, labels, etc. However, since you mentioned you want to have a futuristic user interface (I expect a lot of styling), it seems that all this does not help a lot. In general, I would expect QML to be a better fit to have a fancy user interface. Another thought (though likely a lot more involved) is that Qt has different backends for drawing, e.g. for Window, X Window, macOS, iOS, ... There is also a backend for embedded hardware that does expect a window manager to be running. This last backend could be a basis to write your own backend which draws to a QPixmap (offscreen) which is easily accessible in your game engine.
I don't fully get why you would want to use Qt in this context at all. What you describe will force Qt to work in a way it is not built for. You will always have to fight the normal way Qt usually behaves. Your approach certainly is not supported by Qt at all. Concerning your synchronization of frame drawing sounds a lot like immediate mode UIs. So, why don't you just use IMGUI? BTW, for a futuristic UI you need to do a lot of styling. It would be much easier to just do all the drawing yourself. You say you want to manage button states and mouse positions yourself. I expect you don't want to use any layouts (for automatic resizing). This means that the only thing you are missing in your description is a "collision detection" of your mouse pointer with the buttons. I bet you can easily implement this yourself (or reuse some existing collision detection). Then you have your own fully functioning UI.
The only way Qt would make sense in the context of a game would be to use Qt as usual. There is a way to combine QWidgets with OpenGL which is native to the Qt framework. This would mean your game would have to render to Qt's OpenGL widget. It seems this is not what you want.
@SimonSchroeder Thanks for the long answer! That helps a lot.
I want to use QML because I want to create a modern scifi looking and feeling GUI. Which means a lot of animations. So QWidgers is not the solution I guess.
I t seems creating your own backend is the only solution. But I'm affraind that is outside the scope of my capabilities.
-
@SGaist I think Qt is used for the game editor, isn't it? I want to use Qt for a GUI that is inside the game. Think of a vehicle inside the game that has a dashboard. I'm talking about the GUI on this dashboard.
@Simmania said in How to use Qt for a GUI inside an existing game engine?:
@SGaist I think Qt is used for the game editor, isn't it? I want to use Qt for a GUI that is inside the game. Think of a vehicle inside the game that has a dashboard. I'm talking about the GUI on this dashboard.
Which editor are you talking about ?
You did not give any detail about which engine you are talking about.
