Reactive GUI when rendering is slow – how to?
-
Hello.
When widget rendering is slow, how can a reactive UI be provided?
Starting point
I'm developing a dialog that visualizes the gamut of a given color space. It looks like this:
On the left is a vertical slider that modifies the lightness: A new gamut body (visible within the circle) will be rendered.
The problem is that this rendering is slow. Of course, it depends on your computer and your screen size, but it’s slow. Let’s say 2000 ms. A synchronous approach (together with a cache) is not good: The UI does not feel reactive.
Therefore, I’m using now an asynchronous approach (together with a cache), putting the rendering of the gamut in a thread. The user interface feels more reactive. When you click with the mouse somewhere on the slider, it reacts immediately and shows the slider handle at the new position. And 2000 ms later, when the thread has finished the rendering, the gamut image is updated.
The problem
It is quite possible that the user moves the slider slowly and continuously to see how the gamut changes. In my implementation, whenever a new lightness is set, any current rendering operation is cancelled, and a new one for the new lightness is started. So, when the user moves the slider slowly, let’s say every 500 ms it reaches a new position. Rendering starts. 500 ms later, a new mouse movement event: The rendering, which takes 2000 ms, is still incomplete. It is cancelled, and a new rendering is started. In consequence, while the user keeps moving slowly the slider, there will never be an update to the gamut image. That does not feel correct. But what could be a better solution?
Possible solutions?
I could think of two possible solutions:
-
Remove the code that allows to cancel a currently running rendering. Advantage: Every 2000 ms the user gets a new gamut image (though maybe slightly outdated is he has moved the mouse again in the meantime). Disadvantage: Rendering may take more time when the user releases the mouse button. (The rendering starts at a given moment. 500 ms later, the user makes a last move with the mouse and releases the mouse button. It takes then 1500 ms to finish the previous rendering operation, and another 2000 ms for the final rendering operation: 3500 ms in total.)
-
Whenever the lightness changes, first render a gamut image with poor resolution (rendering time is proportional to the number of pixel, so depending on the resolution this might be very fast), and only then render a gamut image at full resolution.
The question
From a point of view of good UI design, what would be the preferred approach?
-
-
@sommerluk
I know nothing about these graphics, but it doesn't stop me having a firm opinion :)I take it you are saying during the 2 second render nothing happens and then the result appears instantaneously, it can't be done as a gradual thing.
Definitely #2. Moving the mouse, setting off a render which is going to take 2 seconds and knowing it's not the one I desire and wanting to move the mouse again would be too irritating. Given that it's going to take 2 seconds I will settle for a "preview" as I drag around, and a final version when I stop moving. I guess you might set off the full resolution one after, say, half a second of no input. Then if the user does move the mouse again cancel the full 2 second one you had in progress and revert to the quick preview. Till they let you produce the full version without interrupting.
Of course, only you know why this thing takes this long and whether anything can be done to reduce it.
-
Yes, you could setTracking to
falseso that an update is only triggered when the user releases the slider. Also, it sounds like you are already doing a lot of what this example does, but you might want to check out the Mandelbrot example. -
Hi,
Did you try to see the hot paths of your application with a tool like hotspot ?
Beside that, what exactly takes that long in your application ?
From a quick look at your code, you likely do too much in your paint event.
You are creating a new QImage every time paintEvent is called, why not create it once on resize and then just repaint on top of it.
One other possibility is to update the content of the image when something changes and just paint the updated image once that is done.
-
Thanks a lot for these answers.
To summarize:
- Solution 1 from the original post is confusing and therefore not good.
- Solution 2 from the original post is much better.
- An alternative would be to disable tracking of the slider, so the image only gets updated on mouse release.
Disable the tracking would be the easiest thing to do, but it is not very flexible. There are two main problems for me:
- I also have spin boxes in the dialog, which also change the image, and I do not want to disable the arrow-up and arrow-down support for these spin boxes because that seems a bit strange.
- I do not know the speed of the target system, so anyway I prefer to use threads to make sure the UI stays reactive.
Therefore, I think I will go for solution 2.
Stays the question of how to implement this.
Did you try to see the hot paths of your application with a tool like hotspot ?
No.
Beside that, what exactly takes that long in your application ?
From a quick look at your code, you likely do too much in your paint event.
You are creating a new QImage every time paintEvent is called, why not create it once on resize and then just repaint on top of it.Indeed, the paint event (and also the rest of the code) is not speed-optimized at all. My plan was:
- Define which behaviour I want. (That is done now, thanks to you!)
- Decide which threading technologies to use. (To be done.)
- Optimize the individual parts where necessary. (To be done.)
But I remember that, when once ago on a pretty fast computer and with a small widget size, I measured the old synchronous code, the paint event took 360 ms, of which 330 ms were the image rendering algorithm, which has therefore the biggest problem.
Also, I will look into your suggestion to keep always a QImage in memory, so that it is not allocated again on each paint event. (Also, I have heard somewhere that Qt itself does the same thing.)
you might want to check out the Mandelbrot example.
Yes, indeed, the current multithread code is based on the Mandelbrot example.
One other possibility is to update the content of the image when something changes and just paint the updated image once that is done.
Indeed, I'm using yet an image cache, so there is no unnecessary rendering of the image.
Of course, only you know why this thing takes this long and whether anything can be done to reduce it.
That's the most interesting question. The algorithm calculates each pixel independently: Perfect for splitting the work into as many threads as the count of virtual cores of the target computer. And probably, the algorithm itself can also be optimized.
How could this work? I have no experience in multithreading. Doing it manually with QThread seems therefore complicated and error-prone to me. Could I query the number of virtual cores, and then use QtConcurrent::run() as many times as virtual cores are available, each call with just a fraction of the rendering image? Would this scale well? (As I'm developing a library, I have no knowledge about what other threads the application developer will use in his own code.)
it can't be done as a gradual thing.
Actually, probably in can be done as a gradual thing. Maybe allocate a QImage in full-resolution size. Then, calculate pixel (0, 0) and set this color for pixel (0, 0), but also for (0, 1), (1, 0) and (1, 1). And so on, until the QImage is complete. Do a paint event with this low-resolution image. Do a new run and calculate (0, 1), (1, 0) and (1, 1) exactly. And so on, until the QImage is complete. Then, do a new paint event with the full-resolution image. This idea could also be adapted to work with more than only two steps. Do you think this is a good idea?
