Need help with performance of Qt application when X11-forwarding
-
@mrjj I've looked at that documentation, unfortunately its for a different product in their line (we don't have some of the options they mention in that guide). We opened a support ticket with the OpenText Exceed folks to see if we could get some help from them to make sure we've optimized our configuration, hopefully they provide us with something useful in the next week or so. I'm just worried that the issues are more of the client side and not the server. We don't have any flexibility to run using a different product either, we're stuck with them.
Thanks though! :)
-
Did you try a video/youtube or something like that just to see if that runs well ?
Just curious :)
-
@mrjj Each test system is a private network, its just a few machines connected together, and they can't be connected to the internet. I found a QtPerf utility online (designed in Qt4 but works for Qt5 too) that I ran, which cycled through different QWidgets and just draws things rapidly/randomly. I used that to test if the QT_GRAPHICSSYSTEM flag seemed to have any impact, mostly focused on the time taken to perform QImage draws. Thats largely how I determined the flag seemed to have no impact on our performance, and illustrate the performance differences between the two network switches.
-
@ceminger said:
Ah, well that should also generate a lot of screen changes I assume.
I wonder if drawing widgets generates native x calls as for basic primitives where as
updating your bitmap might be sort of one big raster operation.Thank you for the explanation.
I hope the vendor comes up with some tricks.
-
The fact breaking up the image into tiles and only updating the changed ones was beneficial strongly suggest any sort of compression you can bring to bear will be helpful. So:
X11 used to have a couple of compression standards (DXPC and...,I think there was another one too) but it's pretty unlikely a modern system would support them AFAIK. However, that's because the need for them was largely obsoleted by something you should try using if you possibly can...
ssh X11 forwarding. Most non-local use of X11 these days is recommended to be routed through an ssh tunnel for security (all keyboard traffic - typed passwords! - and image data is in-the-clear otherwise). And one of the things the ssh endpoints can do (if you use -C commandline option or have something like "Compression: yes" specified in the ssh config) is compress the X11 traffic before sending it. I do this when using X11 over wifi and it makes a big improvement to graphically intensive applications compared with the non-compressed case. It might seem counter-intuitive to throw more compute power at a networking problem but if spending CPU time at each end compressing/decompressing saves as much transmission time or more... you're ahead, often significantly. Think ssh also has some options for compression level (zlib's 0-9 scale? but maybe an "adaptive" one too) and encryption algorithm (some are a bit faster, but on modern HW encryption is very fast anyway). Sounds like your "tons of blackspace" images could compress quite well and bandwidth would be greatly reduced.
If you're on Windows, cygwin's ssh forwards X11 just as well as it does on Linux. I don't know anything about Exceed but X11-over-ssh is such a common use case I'd be surprised if it didn't have some built in support already.
Finally, re the "Could not initialize OpenGL"... as I understand it, Qt is probably trying to get the Exceed X11 server to talk the OpenGL-over-X11 protocol GLX; it's nothing to do with your application box's drivers. Presumably Exceed doesn't support GLX or OpenGL (any way to enable it?) or good enough OpenGL. If you used another Linux box instead of Exceed you'd probably find it forwards the OpenGL fine; I've certainly seen it working with the Qt C++ QGLWidget (although QQuickView applications fail). This isn't a solution to your bandwidth issues though; GLX doesn't help in that area AFAIK.
-
@timday Thanks! I think we can enable OpenGL on the Windows/Exceed side, so maybe I'll try enabling that just to see if the error goes away. I'm going to see if we can use the ssh tunneling with compression when I get in to the office next week. Will let you know how it goes!
-
@timday Having a team meeting today to discuss the situation. It still looks like the SSH solution is the way to go (we haven't really seen any other feasible ideas on the Qt or Exceed side), I'd at least like to try it and see if the difference is measurable. The implementation though might be a complication for us if it does indeed work... we have to worry about impacting other software thats going to be run that also uses Exceed (so I'm hoping you can use Exceed with and without SSH tunnels at the same time), and it requires us installing additional software on the Windows side, which isn't quite under our control (not sure how that would impact delivery of the final product or our requirements). Also worried about the problems that might be introduced if our shh tunnels disconnect but our program is still running (I think we need to implement this in a way where the tunnels are created by scripts initially and then our Qt app is started and stopped on the fly). Need to be able to handle these situations gracefully. Anyways, thats where we are at today. Still soliciting solutions from others if they have any ideas!
-
@ceminger The way the ssh X11 forwarding works, there's no reason I know that an X11 "server" shouldn't be able to display a mix of content from both ssh-tunneled X11 applications and ones connected the normal way (hmmm in fact the display I'm currently typing this on contains a mixture of exactly that in fact). Similarly at the X11 "client" (application) end, whether an application is forwarded or not is simply controlled by pointing the DISPLAY environment at the tunnel's localhost endpoint and so can be done on a process by process basis).
It occurs to me there's another possibility: OpenVPN (and probably other VPN software) has some compression options which attempt to compress all traffic over the virtual network (OpenVPN options --comp_lzo and --comp_noadapt). Whether that's a better solution or not probably depends whether hoovering up all network traffic between 2 machines is seen as a useful convenience or as not allowing sufficiently fine grained control.
You might find NX technology https://en.wikipedia.org/wiki/NX_technology of interest too, although in the high-latency WAN space it mainly seemed to be targeting, I get the impression more screen-scraping based approaches have emerged victorious.
-
@timday Not so sure about OpenVPN, but you mentioned NX and someone else in our organization brought it up as well, so we may have to give that some consideration. Currently working on setting up an SSH server on Windows so we can test that out, not sure if they had settled on one where the licensing was agreeable (for some reason freeSSHd, which looked like the best option to me, was not). In the meantime one of the Exceed support folks suggested trying to turn on PsuedoColor emulation to force it a 256 color palette, which actually would work for us if we are able to use a custom palette that we get to define... Well I tried that option and our Qt app did not play nice with it, the application appeared to be invisible at that point, and all we could get is a view of the windows we were overlaying, with their color depth reduced. I could click on buttons I couldn't see to bring up a dialog, but it had the same issue, where none of the Qt stuff seemed to actually be drawn.
Changing the color depth seems like it should be a viable solution, but I have no idea what effort we have to make on the Qt side to make this happen. I may open up another thread specific to that issue, but I don't have high hopes since there's pretty scant info on using QApplication::setColorSpec and how it works with x11.
-
The 8 bit paletteized option (PseudoColor in X11 speak I think) is an interesting idea. Last time I used that stuff was late 80s when true RGB displays were much more expensive; I was glad to see the back of it. We did indeed used to use a custom palette depending on what we were displaying: for greyscale images we used some palette with something like 224 greys and the rest for a few "UI colors"; for color images we had to dither to 3 bits red, 3 bits green and 2 bits blue and it still looked terrible. No idea whether Qt supports such customization; documentation at http://doc.qt.io/qt-4.8/qapplication.html#setColorSpec suggests Qt has it's own palette built-in for such purposes. I'd be amazed if anyone was seriously using this sort of palletized graphics display these days when it seems OpenGLES is defines the min.spec graphics for even the lowliest embedded chips.
But if you think the 3:1 or 4:1 reduction in bandwidth moving from a 24/32-bit "visual" to an 8 bit one is all you need, AND most of your bandwidth hogging content is greyscale images, then my instinct would be to just look into displaying the images by sending them to the server (display) side using a single channel OpenGL GL_R8/GL_INTENSITY8 texture, and appropriate GL commands to texture that to a view-filling polygon. This changes the solution focus a bit from messing around with Qt to messing around with OpenGL though.
