Qt Windows application deployment issue. Runs great in Debug, but not fully functional as deployed, stand-alone app

MarkMcC

Greetings,
I have an application I have written in Qt Creator that I have fully debugged in the Qt IDE and when I make a stand-alone release version, that verion only works partially. Here is a very brief rundown on the app, what works, and what doesn't.
The Main body of this is a UI full of tables which the user can edit. These edited tables can be then written to disc files, and previously saved tables can be retrieved from disc files to update the UI. The tables in the UI can also be written to the EEPROM of an Arduino NANO board through the USB port. Files may also be retrieved from this external EEPROM, and the results displayed in the editable UI. All these function work perfectly when running in DEBUG within the Qt Creator IDE.
I then generated a release build of the code and performed the WINDEPLOYQT.EXE on the release build. The build was successful and the application now launches outside of the Qt environment. I now have a stand alone app with a UI which is responsive to edits. I can save the edited tables to disc files, and I can retrieve previously saved files into my UI. All good so far, but when I attempt to read or write to the external EEPROM though USB, the app behaves as if the port is not there. As an aside here, the EEPROM communication is performed by an external application called AVRDUDE. For completeness here, I include the snippet from my source which executes this.

void MainWindow::call_AVRDUDE_write()   //AVR WRITE
{
    QProcess CommandPrompt;
    QStringList Arguments;
    QString COMPortUsed = (ui->COM_Port_Used->text()); // get the COM port used from the UI
    Arguments << "/C avrdude -c arduino -P " + COMPortUsed + " -b 115200 -p ATmega328P -e  -U eeprom:w:tblXfer.bin:r";
    CommandPrompt.execute("cmd",Arguments);
}

There is also a similar Read function I've written, and again, both work in DEBUG, but neither work in the deployed release.
There is nothing different is the source code or hardware between the DEBUG version (successful) and the DEPLOYED version (broken) that I can imagine. I'm wondering if there is something missing in the package that WINDEPLOYQT.EXE put together in this released version. Below is an excerpt of the verbose command line script generated during the WINDEPLOYQT.EXE pass

C:\Users\trujo\Documents\build-TablesPrototype-Desktop_Qt_6_0_4_MinGW_64_bit-Release\release\TablesPrototype.exe 64 bit, release executable
Adding Qt6Svg for qsvgicon.dll
Direct dependencies: Qt6Core Qt6Gui Qt6Widgets
All dependencies   : Qt6Core Qt6Gui Qt6Widgets
To be deployed     : Qt6Core Qt6Gui Qt6Svg Qt6Widgets
Updating Qt6Core.dll.
Updating Qt6Gui.dll.
Updating Qt6Svg.dll.
Updating Qt6Widgets.dll.
Updating opengl32sw.dll.
Updating D3Dcompiler_47.dll.
Updating libgcc_s_seh-1.dll.
Updating libstdc++-6.dll.
Updating libwinpthread-1.dll.
Creating directory C:/Users/trujo/Documents/build-TablesPrototype-Desktop_Qt_6_0_4_MinGW_64_bit-Release/release/iconengines.
Updating qsvgicon.dll.
Creating directory C:/Users/trujo/Documents/build-TablesPrototype-Desktop_Qt_6_0_4_MinGW_64_bit-Release/release/imageformats.
Updating qgif.dll.
Updating qico.dll.
Updating qjpeg.dll.
Updating qsvg.dll.
Creating directory C:/Users/trujo/Documents/build-TablesPrototype-Desktop_Qt_6_0_4_MinGW_64_bit-Release/release/platforms.
Updating qwindows.dll.
Creating directory C:/Users/trujo/Documents/build-TablesPrototype-Desktop_Qt_6_0_4_MinGW_64_bit-Release/release/styles.
Updating qwindowsvistastyle.dll.
Creating C:\Users\trujo\Documents\build-TablesPrototype-Desktop_Qt_6_0_4_MinGW_64_bit-Release\release\translations...
Creating qt_ar.qm...
Creating qt_bg.qm...
Creating qt_ca.qm...
Creating qt_cs.qm...
Creating qt_da.qm...

(There were several more lines here, but I'm pretty sure they are just more language translation items so I omitted their inclusion here.)

So what gives? I did not see any errors in the build, nor in this script when WINDEPLOYQT was executed (above). Did something fail to get included here which was present when I was running in Qt Creator DEBUG mode? Any thoughts on how I might chase this down? Regards...Mark

JKSH

@MarkMcC Some thoughts:

Perhaps your Debug app is still holding onto the COM port? Try rebooting and running the deployed app without touching the Debug app.
What happens when you run the Release build from the Qt Creator IDE?
How does your app know where to find avrdude?

MarkMcC

@JKSH Thanks for the input. I did go through your suggestions/questions. Here is where things stand.

rebooting system and running deployed app away from IDE does not resolve problem.
If I run the release build out of the IDE, it works fine.
AVRDUDE is in the environment variables path, just as it is for the DEBUG version in the IDE. Just as a thing to try, I have also copied the AVRDUDE folder contents into to the local directory where the deployed version of my program lives. Still no soap.

J.Hilk

@MarkMcC might be a permission issue, QtCreator passes its own permissions to all processes it spawns, that usually also includes apps you build and start.

JonB

@J-Hilk said in Qt Windows application deployment issue. Runs great in Debug, but not fully functional as deployed, stand-alone app:

might be a permission issue, QtCreator passes its own permissions to all processes it spawns

Hi there. What exactly do you mean by this, can you give an example, please? And how would it have access to "different permissions" anyway, given that Creator is run with same permissions as any other process I spawn, including any app being built (e.g. if I ran that outside Creator)? It might e.g. set different environment variables, but that's quite different.

mchinand

@MarkMcC said in Qt Windows application deployment issue. Runs great in Debug, but not fully functional as deployed, stand-alone app:

tblXfer.bin

Does this file exist and is it where your QProcess/avrdude expects it? If you are creating it in your application, is it writing to a location that your application can write to?

MarkMcC

Well, I found a BIG lead in this issue, unfortunately t doesn't make sense to me what the relationship is between working in the IDE versus NOT working in deployment. My amended calling code to the external process (AVRDUDE) is as shown below. Note that my instance of QProcess now uses "startDetached() rather than "execute()".

void MainWindow::call_AVRDUDE_read()    //AVR READ
{
    QProcess CommandPrompt;
    QStringList Arguments;
    QString COMPortUsed = (ui->COM_Port_Used->text()); // get the COM port from the user off UI
    Arguments << "/C avrdude -c arduino -P "+ COMPortUsed +" -b 115200 -p ATmega328P -e  -U eeprom:r:fromEEPROM.bin:r";
    CommandPrompt.startDetached("cmd",Arguments);
}

This modified code now runs under both QT Creators' IDE and as a stand alone deployment! Unfortunately, the use of "startDetached()" returns control to the calling process prior to the completion of its task. AVRDUDE goes out and reads an external EEPROM over USB , which takes a few seconds. Control is returned immediately though to the calling function. In my code running under DEBUG in the IDE (prior to attempting deployment) I resolved this by using "execute()" which contains a waitForFinished embedded in the function. Also, I originally tried just QProcess::start(), but that did not work for me either.

So, for some mysterious reason, QProcess seems to behave differently in the IDE versus in deployment for me. Using "startDetached()" works in the sense that it spawns the external AVRDUDE process successfully, but I can't tolerate the premature return of control to my calling process.
What this DOES show however, is that there is something rotten going on here with this QProcess call where it behaves DIFFERENTLY in IDE versus as a stand alone deployment. "startDetached() " DOES in fact launch my external AVRDUDE script and it DOES run it properly. It is it's premature return that is causing grief in its use alone as a "solution" to this mystery.

Is there something going on under the hood in QProcess that can be addressed, or at least understood? And why the difference between debugging in IDE versus deployed version?

JonB

@MarkMcC said in Qt Windows application deployment issue. Runs great in Debug, but not fully functional as deployed, stand-alone app:

debugging in IDE versus deployed version?

Are you saying that you are running from IDE under debugger, and comparing that against deployed version running freely outside of any debugger? When you run your program from Creator IDE, what button/command do you use? I assumed you were using "Run" not "Debug".

One other, separate thing: is this avrdude a .bat or a .exe file? Do you have to run your command line via cmd /c, that just complicates things and waiting for it to terminate.

mchinand

Is there a reason why you are running it as cmd.exe /C avrdude ... instead of avrdude directly?

MarkMcC

Hi again Jon! When I am in the qt Creator IDE, I can run successfully using either "Run" or "Debug" (QProcess::execute() in place in the code). I can also build in either Debug or Release mode, and everything is fine. In the IDE
BTW, sorry I was not complete in my description of my setup. I am running Qt version 6.0.4 , Mingw_64bit in my kit, and under Windows 11.

JonB

@MarkMcC
I misunderstood, I thought the deployed version worked fine but not from IDE. I now see you are saying the opposite.

Although it may seem a different issue, both @mchinand & I asked whether you can execute avrdude directly without going via cmd /c? That might simplify matters a bit, and would seem preferable anyway if it works.

JonB

@MarkMcC
Going back to QProcess. If you want to wait for the process to finish you will need to use QProcess::start(). startDetached() won't let you wait.

QProcess::execute() convenience function hides the QProcess from you. For a start you should check its return value, for all I know it is indicating an error occurred. If you change to QProcess::waitForFinished() at least you can check for anything it might have written to stdout/err. Maybe avrdude is writing you a message? Best would be signals/slots, methods of QProcess, you just have to keep the instance in scope, say a member variable. I would expect to write it that way if I needed to examine what might be going on from here.

MarkMcC

@mhinand and @JonB
Yes. I had last week I originally wrote the code without the "/C" prefix and my program hung at the QProcess call. I also could see that the activity LED on my dongle at the end of the USB cable was not indicating any traffic, so it too failed to launch.
With the leading "/C", all works. In fact my code above where I replaced "execute" with "startDetached", The argument passed also includes the "/C" prefix.

@JonB Sorry about the general confusion. The deployed version is the version which did NOT work with QProcess::execute(), the IDE version does.

JonB

@MarkMcC said in Qt Windows application deployment issue. Runs great in Debug, but not fully functional as deployed, stand-alone app:

With the leading "/C", all works. In fact my code above where I replaced "execute" with "startDetached", The argument passed also includes the "/C" prefix.

This is not what we are asking you to do. We want to eliminate cmd from the problem.

Does anything work at all if you try:

    Arguments << "-c" << "arduino" << "-P" << "COMPortUsed" << "-b" << "115200" << "-p" << "ATmega328P" << "-e" << "-U" << "eeprom:r:fromEEPROM.bin:r";
    int exitCode = CommandPrompt.execute("avrdude",Arguments);
    qDebug() << exitCode;

MarkMcC

@JonB ..back at it. I tried the changes above you recommended, and unfortunately, it did not launch AVRDUDE. The exitCode I got back was a "1", measured at a breakpoint I put on the qDebug() << exitCode line. Also, at this point, I examined the "Arguments" QStringList, and I have 11 separate elements there. In mine, I have one contiguous string, delimited by spaces. If I check the exit code after execution, it is a "0".
Also, could you elaborate on your above comment

Best would be signals/slots, methods of QProcess, you just have to keep the instance in scope, say a member variable. I would expect to write it that way if I needed to examine what might be going on from here.

I don't follow what you are saying there as it is I'm afraid above my level of c++ programming knowledge. I'm happy to try anything though.

I still find the core of all this, the disconnect between Deployed version versus the running in IDE version pretty baffling. If it was just some screwy code I've written, why would it work under the IDE and then not in the deployment?

mchinand

What is the output of readAllStandardError() and readAllStandardOutput() when the QProcess fails?

MarkMcC

OK, here is a bit of code I came up with that now works in both in the IDE environment and in the deployed version.

void MainWindow::call_AVRDUDE_write()   //AVR WRITE
{
    QProcess CommandPrompt;
    QStringList Arguments;
    QString COMPortUsed = (ui->COM_Port_Used->text());     // get the COM port used from the UI
    Arguments << " /C avrdude -c arduino -P " + COMPortUsed + " -b 115200 -p ATmega328P -e  -U eeprom:w:tblXfer.bin:r";
    CommandPrompt.start("cmd",Arguments);
    CommandPrompt.waitForFinished();
}

The last two lines replace the QProcess::execute() line I had earlier that worked only in IDE. What bothers me is that I had tried start() by itself before and it did not work for me, even in IDE environment. I previously tried prefacing it with

CmmandPrompt.setProcessChannelMode(ForwardChannels);

and that seemed to allow me to run start(), and subsequently waitForFinished();
Now I find that actually removing that allows for successful operation
in both IDE and Deployment renditions. Bizzare! This is disturbing because it almost seems like capricious behavior, and I don't like the fact that the change makes such little sense to me. The "problem" seems to be resolved, but I don't have a clue as to why this worked, or for that matter, why there was a disconnect between IDE operation and deployed operation in the first place. @mchinand and @JonB ,Thanks guys for your help. I realize I'm still chasing my tail until this is understood but at least it works (for now). Mark

JonB

@MarkMcC
Some answers to your questions.

I tried the changes above you recommended, and unfortunately, it did not launch AVRDUDE. The exitCode I got back was a "1", measured at a breakpoint I put on the qDebug() << exitCode line.

That was always a possibility, and what I was trying to investigate. It implies that avrdude is not an executable and has to be invoked via cmd /c. Which is OK, but complicates matters and makes it harder to know what is going on.

Also, at this point, I examined the "Arguments" QStringList, and I have 11 separate elements there. In mine, I have one contiguous string, delimited by spaces.

That is correct. To launch an executable directly via QProcess all its command-line arguments must be sent as separate items in a QStringList. Hence I broke up your command on spaces and separated out all the arguments for you to copy the line. It is only via cmd /c that one can present the whole command line as a single string instead of separate arguments.

If I check the exit code after execution, it is a "0".

One of the disadvantages of running a command via cmd /c is that you will always get an exit code of 0 rather than whatever the command being run actually returns as its exit code.

Also, could you elaborate on your above comment
I don't follow what you are saying there as it is I'm afraid above my level of c++ programming knowledge.

I was asking you to rewrite the QProcess call using the signals/slots approaches it offers. That is the low-level on which other QProcess calls like execute() and waitForFinished() is built, and offers us more control/knowledge over what is going on. However for now at least given that you cannot write it I don't have the time/inclination to write it for you to copy.

The last two lines replace the QProcess::execute() line I had earlier that worked only in IDE. What bothers me is that I had tried start() by itself before and it did not work for me, even in IDE environment. I previously tried prefacing it with
CmmandPrompt.setProcessChannelMode(ForwardChannels);
and that seemed to allow me to run start(), and subsequently waitForFinished();
Now I find that actually removing that allows for successful operation
in both IDE and Deployment renditions. Bizzare!

Now this is interesting! You say it only works correctly when you do not have CommandPrompt.setProcessChannelMode(ForwardChannels). The utility function QProcess::execute() you were previously using does put that line in. If that must not be present, it would explain why your code now differs and works versus using execute(), so at least that is something.

You should do one thing. Immediately after your CommandPrompt.waitForFinished(); you should see whether any messages might have been written to stdout/err by the subprocess. Append these lines:

qDebug() << CommandPrompt.readAllStandardOutput();
qDebug() << CommandPrompt.readAllStandardError();

I do not use Windows. I am told that it is difficult to see any output from qDebug() when run outside the IDE? If that is so you might have to replace the qDebug() by something like

std::cout << CommandPrompt.readAllStandardOutput() << std::flush;
std::cout << CommandPrompt.readAllStandardError() << std::flush;

(You will need #include <iostream> at the head of the .cpp file.)
Since you say it now actually works without CommandPrompt.setProcessChannelMode(ForwardedChannels);, you may need/want to put that back in when seeing whether there might be any error messages output.

I still find the core of all this the core of all this, the disconnect between Deployed version versus the running in IDE version pretty baffling. If it was just some screwy code I've written, why would it work under the IDE and then not in the deployment?

I agree. If I knew what is happening outside the IDE which is different from inside the IDE I would say so!

MarkMcC

@JonB Thanks Jon. I will look into those suggestions. I think at this point though, the best thing for me to try, rather than thrashing around with my current project, would be for me to start a new project focused on just replicating this specific issue. I'll try and pare this down to minimal required code, just enough to reproduce the problem This will make findings clearer and easier to communicate to others. If I get to a point where I have something useful to share, I'll do so. Thank you again..Mark