Can't read from the standard input

Adar70

Hello. I have this code here:

import sys
from PySide6.QtCore import QCoreApplication, QProcess, QByteArray


class ProcessExample:
    def __init__(self):
        self.process = QProcess()
        self.process.readyReadStandardOutput.connect(self.handle_stdout)
        self.process.readyReadStandardError.connect(self.handle_stderr)
        self.process.finished.connect(self.handle_finished)

    
    def start_process(self):
        self.process.start("python", ['test_qprocess_program.py'])


    def write_to_process(self, message):
        print("No. bytes written: ", self.process.write(message))
        self.process.waitForReadyRead()


    def handle_stdout(self):
        stdout_output: QByteArray = self.process.readAllStandardOutput()
        print("Standard Output:")
        print(stdout_output)
        print(stdout_output.data())
        print(stdout_output.data().decode('utf-8'))
        out = stdout_output.data().decode('utf-8').splitlines()
        print(out)
                
    
    def handle_stderr(self):
        error_output: QByteArray = self.process.readAllStandardError()
        print("Standard Error Output:")
        print(error_output)

    
    def handle_finished(self):
        print("End")


app = QCoreApplication(sys.argv)
p = ProcessExample()
p.start_process()
p.write_to_process(QByteArray(bytearray((0x03, 0x01, 0x23, 0x45, 0x67))))
app.exec()

And this code in the test_qprocess_program.py:

b = sys.stdin.read()
print(b, flush=True)

It's supposed to give me those bytes (0x03, 0x01, 0x23, 0x45, 0x67) that I write to it in some way, but it just does nothing. There is no echo. I can't understand why. I also tried b = sys.stdin.buffer.read() and QByteArray(bytearray((0x03, 0x01, 0x23, 0x45, 0x67))).toHex(). Please help me.

JonB

@Adar70
EOF is not a character you can send, it's a state/condition, which is true when end of file is detected. For a process sending the data (as opposed to reading from a file) that requires either to terminate/exit the sending process (your QProcess) --- which will mean it won't be there to read the response back in your case --- or you must close stdout, QProcess::closeWriteChannel() --- which will mean you won't be able to send any further messages to the subprocess, which may or may not be acceptable in your situation.

However, better, I will bet my bottom dollar that Python has a way of reading from stdin which does not require EOF. That is down to your statements like sys.stdin.read(), which is a Python question. You want some kind of "unbuffered" and "return immediately with what is there". That is for you to look up. I don't think you should read from sys.stdin.buffer, that is likely to be buffered. Why don't you start with os.read(0, 5) --- read 5 bytes from file descriptor 0, which is stdin, and should be unbuffered --- and see if that works?

Failing that you might try sending a \n at the end of your message, if Python's stdin is line-buffered that may work. If you had sent the data from Pythion via print(..., flush=True) that would have added a \n and I believe that would allow reader to use e.g. for line in sys.stdin (or equivalent for one line) and receive the line immediately. However, your data looks like arbitrary binary data, not text, so the whole idea of inserting a \n or reading "lines" may not be appropriate here.

You might also temporarily try writing the sender side in Python too while you get it working.

SGaist

Hi,

I don't have a machine at hand but you don't wait for the process to be started. It's asynchronous.
Also, why are you reading the buffer of stdin rather than stdin itself ?

Adar70

@SGaist

I don't have a machine at hand but you don't wait for the process to be started. It's asynchronous.

So are you saying that the process is not up yet? When I was sending from the process (not receiving and then sending) it worked fine.

Also, why are you reading the buffer of stdin rather than stdin itself ?

What do you mean? I tried both sys.stdin.buffer.read() and sys.stdin.read(), both failed.

SGaist

The thing is you call start and right after your write something to the stdin however there's no guarantee that your script has started at this point.
You should use the started signal to then write to the process and you should also connect the errorOccurred signal because you never know what might go wrong.

Adar70

@SGaist
I've updated the code for the process class:

import sys
from PySide6.QtCore import QCoreApplication, QProcess, QByteArray


class ProcessExample:
    def __init__(self):
        self.process = QProcess()
        self.process.readyReadStandardOutput.connect(self.handle_stdout)
        self.process.readyReadStandardError.connect(self.handle_stderr)
        self.process.errorOccurred.connect(self.handle_process_error)
        self.process.stateChanged.connect(self.current_process_state)
        self.process.finished.connect(self.handle_finished)

    
    def start_process(self):
        self.process.start("python", ['test_qprocess_program.py'])
        self.process.waitForStarted()


    def write_to_process(self, message):
        print("No. bytes written: ", self.process.write(message))
        self.process.waitForReadyRead()


    def handle_stdout(self):
        stdout_output: QByteArray = self.process.readAllStandardOutput()
        print("Standard Output:")
        print(stdout_output)
        print(stdout_output.data())
        print(stdout_output.data().decode('utf-8'))
        out = stdout_output.data().decode('utf-8').splitlines()
        print(out)
                
    
    def handle_stderr(self):
        error_output: QByteArray = self.process.readAllStandardError()
        print("Standard Error Output:")
        print(error_output)

    
    def current_process_state(self, state):
        print(state)


    def handle_process_error(self, error):
        print('Process error:')
        print(error)


    def handle_finished(self):
        print("End")
        #self.process.kill()
        #print('3')


app = QCoreApplication(sys.argv)
p = ProcessExample()
p.start_process()
p.write_to_process(QByteArray(bytearray((0x03, 0x01, 0x23, 0x45, 0x67))))
app.exec()

Still nothing. The console shows

ProcessState.Starting
ProcessState.Running 
No. bytes written:  5

forever.

Adar70

I believe I know the issue. Python's sys.stdin.read() expects EOF at the end of the message, which QProcess.write() doesn't provide. Any workaround or an actual fix?

JonB

@Adar70
EOF is not a character you can send, it's a state/condition, which is true when end of file is detected. For a process sending the data (as opposed to reading from a file) that requires either to terminate/exit the sending process (your QProcess) --- which will mean it won't be there to read the response back in your case --- or you must close stdout, QProcess::closeWriteChannel() --- which will mean you won't be able to send any further messages to the subprocess, which may or may not be acceptable in your situation.

However, better, I will bet my bottom dollar that Python has a way of reading from stdin which does not require EOF. That is down to your statements like sys.stdin.read(), which is a Python question. You want some kind of "unbuffered" and "return immediately with what is there". That is for you to look up. I don't think you should read from sys.stdin.buffer, that is likely to be buffered. Why don't you start with os.read(0, 5) --- read 5 bytes from file descriptor 0, which is stdin, and should be unbuffered --- and see if that works?

Failing that you might try sending a \n at the end of your message, if Python's stdin is line-buffered that may work. If you had sent the data from Pythion via print(..., flush=True) that would have added a \n and I believe that would allow reader to use e.g. for line in sys.stdin (or equivalent for one line) and receive the line immediately. However, your data looks like arbitrary binary data, not text, so the whole idea of inserting a \n or reading "lines" may not be appropriate here.

You might also temporarily try writing the sender side in Python too while you get it working.

Adar70

@JonB
Thanks for help. I experimented with sys.stdin.read(5) and it works. The data doesn't have to be binary, I can convert it, so using QTextStream, adding a trailing newline character and receiving with sys.stdin.readline() works as well.