QProcess not printing "live update" from cmd

[Dariusz]

Hey

I have QProcess runing sftp application on windows. That app uploads data and the "status" is being send back as progress.

QProcess ignores these messages and I cant read them. How can I configure qprocess to get that print status?

When I use ForwardChannels I can see in my console % update, but neither readyReadOutput/Error gets triggerred.

Any ideas?

Regards
Dariusz

[JonB]

@Dariusz
If sftp writes to stdout or stderr then QProcess can capture it. If it does not, then it cannot. So if you are not getting these "status messages" (even by the time the process ends if perhaps it buffers them rather than immediately flushing) they are not being sent to either of those channels. One possibility is that sftp only sends them if it detects the output is a terminal (e.g. int _isatty( int fd )) ...

[Dariusz]

@JonB How can I flush/read from buffer? Can I force/manual read it?

[JonB]

@Dariusz
No, that happens at the outputting process side (sftp). But in this case you would still receive all the output, just it would be at the end of the QProcess not as it runs. Unless you see this does happen there is no point investigating it further.

Try running your sftp from a Command Prompt but with all its output redirected to a file. When it has finished what is in that file, any "status messages"?

EDIT
Oh I see, you mean by

When I use ForwardChannels I can see in my console % update, but neither readyReadOutput/Error gets triggerred.

that you do see status messages arriving? Then you should get readyRead... signals. But (quite possibly?) not if you use QProcess::ForwardedChannels, if that is what you are saying, why would you do that if you want to "capture" the output?

[Dariusz]

@JonB I tried ussing readyRead, sadly no signal trigger.

Also tried forwarding/merging/etc channels. None of the channel mode trigger any of signals.
If I forward I can see it in the console.
Esentially its FilePath % mb and the % and mb updates. It does not print new lines per each %. It just replace existing text.

I've no idea how they can replace print in my cmd window but thats what they do. That's what I want to capture as I want to know % of upload.
But since its not a new line, or its some kind of "live buffer" nothing in Qt triggers/tells me that data in buffer changed. Its bananas.

Its like progress bar message. Inline update. No new lines.

Ok we figured some of it out!

Its "updatge %10 mb100 \r" ! the "\r" makes it reprint in the same line again. But there is no \n so there is no end of message and it comes as endless messsage? I've no idea. sigh.

[JonB]

@Dariusz
None of this answers the questions.

• Remove all forwarding/merging, just slot onto readyReadStandardOuptut & readyReadStandardError, and finished.
• Once again, answer whether you get any of this when the QProcess has terminated? I think I have mentioned this twice before.
• Show minimal code, so I can verify you're doing it right.

[Dariusz]

@JonB

• Remove all forwarding/merging, just slot onto readyReadStandardOuptut & readyReadStandardError, and finished.

Yup tried that, no triggers.

• Once again, answer whether you get any of this when the QProcess has terminated? I think I have mentioned this twice before.

Just tried it, nope, nothing comes in when app terminates / we ask it to quit. The sftp does not terminate at end of command. it stays open and wait for next command.
This is not 1 line command. We start app with credentials, then we send message to it ta o execute specific action.

• Show minimal code, so I can verify you're doing it right.

        self.process = QProcess()
        self.process.setProgram("sftp")
        self.process.setArguments(["-i", "C:/.ssh/key", "testFTP@192.168.0.10"])

        #self.process.setProcessChannelMode(QProcess.ForwardedChannels)

        def sig():
            print( "out:",self.process.readAllStandardOutput())
            print( "err:",self.process.readAllStandardError())
            print( "all:", self.process.readAll())


        self.process.readyReadStandardOutput.connect(sig)
        self.process.readyReadStandardError.connect(sig)

        self.process.start()
        self.process.write('put "C:/FS_2022.mov" "/sftp_transfert/FS_2022.mov" -o\n'.encode())

once write with put get executed. We start getting back progress update in terminal. None of it comes to qprocess signals :- (

[jsulm]

@Dariusz said in QProcess not printing "live update" from cmd:

self.process.start()
self.process.write('put "C:/FS_2022.mov" "/sftp_transfert/FS_2022.mov" -o\n'.encode())

You should wait until writing is possible after starting process: https://doc.qt.io/qt-5/qprocess.html#waitForStarted or https://doc.qt.io/qt-5/qprocess.html#started

[JonB]

@Dariusz said in QProcess not printing "live update" from cmd:

The sftp does not terminate at end of command. it stays open and wait for next command.

Then that could be your problem. If lines output not getting flushed you won't get them in real time. Try sending your sftp whatever "command" it requires to tell it to finish, e.g. quit or bye or close its input, I don't know. At the rate you are going that is the only way to be sure we see all its output. Or, send it lots of put commands, so that you make it produce voluminous output to (hopefully) exceed any buffer. Or, send it some commands we know produce output --- doesn't it accept like dir or list, things like that?

[Dariusz]

Hey

waitForStarted does not help sadly.
Sending another message after put does not help either, as if put takes 4h to upload we won't know what % is the upload at.

Its odd, seems like no1 tried to use native windows sftp cmd client to upload data and get % update from it.

Its crazy frustrating, there is my friend & I on it now for 2 days and we cant get it to print/trigger progress update signals :- )

[JonB]

@Dariusz

@JonB said in QProcess not printing "live update" from cmd:

Or, send it some commands we know produce output --- doesn't it accept like dir or list, things like that?

@Dariusz said in QProcess not printing "live update" from cmd:

Sending another message after put does not help either, as if put takes 4h to upload we won't know what % is the upload at.

Do you understand that we/you are supposed to be discovering just what is going on with the output? Not a final solution yet....

Anyway if you don't want to try stuff that's up to you. Hope someone will know just what you need to do without experimenting.

[Dariusz]

@JonB uh sorry I though I was clear

if we do

process.write(put)
process.write(ls)

then we will get:
put = we get put out in - last message, not 1/2/3/4/5/6/7% updates.
ls = list of items.

So ls will give us message of items in dir, and put will just echo command we send to it back to us. But no update. We will get 2 signals in total.

I found out the app that we are using, apparently its not windows sftp, its openSSH sftp client.

[JonB]

@Dariusz
So you know the redirection is working. At least probably on stdout, don't know about stderr. I have tried to explain the possibilities about the messages. From what you have shown/tested so far I am not convinced it sends any update messages when the output is not attached to a console, I suggested things you could try earlier on to determine whether this might be the case.

[Dariusz]

There was no signal with any errors.

as far as I can understand, sftp sends messges like

"fileProgress 1% \r"
"fileProgress 2% \r"
"fileProgress 3% \r"
"fileProgress 4% \r"
"fileProgress 5% \r"
"fileProgress 6% \r"
Because there is no \n, the qt does not trigger any signals. Not byteavailable or anything really as far as I can tell.

Um

• I am not convinced it sends any update messages when the output is not attached to a console

That sounds interesting, could be true yes! I take when we self.process.setProcessChannelMode(QProcess.ForwardedChannels) he then get console somehow?

• I suggested things you could try earlier on to determine whether this might be the case.

Can you link me to post/text? I'm blind :- (

[JonB]

@Dariusz said in QProcess not printing "live update" from cmd:

sftp sends messges like

But (so far as I can see) you know it does that only when output is attached to a terminal/console. Which their code may be testing for, and not sending if not.

Because there is no \n, the qt does not trigger any signals. Not byteavailable or anything really as far as I can tell.

The presence or absence of \n will not itself affect whether Qt/QProcess sees or receives bytes. It might be an indication that it is not being flushed immediately, but that is a different matter.

I take when we self.process.setProcessChannelMode(QProcess.ForwardedChannels) he then get console somehow?

I believe that attaches the QProcess's output to the output of the Qt program, which is a console/terminal. Which as I say may be why in that case, and only that case, you see the output. But then you won't be able to "capture" it.

Can you link me to post/text? I'm blind :- (

Principal among these was

Try running your sftp from a Command Prompt but with all its output redirected to a file. When it has finished what is in that file, any "status messages"?

You should test something like:

sftp < file_of_input_commands > some_file

If you can also redirect Windows/DOS/cmd stderr that would be even better. Like under Linux it would be

sftp < file_of_input_commands > some_file 2>&1

but I can't recall whether/how you can do that 2>&1-type thing with Windows. Test without that anyway.

The point here is that output should not be left to go to console. This is close to simulating what is going on when you have another program --- Qt or not --- running a sub-process with redirection. I have a hunch this will show that in this situation you do not get the "update" messages, because sftp program detects the output redirection and does not issue them in this case.....