Why don't the signals emit from QThread ?

BjornW

Its cool. I'm not saying your code is bad, I'm just saying it is not very easy to jump into someone elses code, regardless how good it is.

Now, if you create a thread, and by that i mean just a thread. No buffers and stuff, and create some object to emit a signal, is it "late"? I suppose not, after that, just keep adding stuff until you can actually reproduce the "late" behavior with the absolutely least amount of code. It is a good idea, even for your own sake, to try to resolve the issue like that. It is very easy to get lost in other workings of your code.

I'd like to try the following:

In your DelayedBuffer you create a timer like so:

L50: self.timer = QTimer()

This timer is not a child of the DelayedBuffer and therefore will not move with the buffer into a new thread. This means it will be executing in the thread where it was created (?). Try changing to

L50: self.timer = QTimer(self)

Otherwise:
Have you tried flushing your outputs?
Why do you call processEvents?

kshegunov

@daegontaven said in Why don't the signals emit from QThread ?:

Okay but PythonInterpreter is already inside it's own thread as shown here. Does it not apply to self.stream because of the multiple inheritance ? Or am I missing something ?

That a QObject belongs to a thread is something Qt introduces. In regular C++ there's no such thing - the data (object) is separate from the thread the methods are run in. In your case the constructor of PythonInterpreter is run in the same thread as the one the calling function is run in, meaning everything you do in that constructor is again in the same thread. As you're creating an object in said constructor that object is going to be in the same thread, understand where I'm getting with this? If you later call moveToThread that applies only to the one object, not to all the objects created in the constructor. So to make this work properly Qt has introduced object ownership - i.e. one object is a child to another object. If an object's moved to a thread all its children are moved to the same thread. However you don't set a parent to the object you create in the constructor so those objects have no parent, thus are not children to the PythonInterpreter instance, and finally are not going to be moved to the PythonInterpreter's object's thread. There are 2 things that can be done to solve this:

Always set the parent for any QObject instance you create, so you have a nice tidy and predictable object tree.
Move all the objects manually to the required thread, which is quite cumbersome.

More on ownership and trees you can find here and QObject::moveToThread documentation specifically refenreces which objects are moved and what are the limitations of this function.

daegontaven

@kshegunov I updated the gist. Is this how you set a parent ?

Or do I have to call setParent ? Which object do I call that method on ?

kshegunov

self.stream = ConsoleStream(this)

would be my guess (I don't know python, so you need to adjust that line accordingly).

Which object do I call that method on ?

For each QObject derived class' instance you want to be moved with the parent to the thread. As is now, the PythonInterpreter instance is moved to a thread, here:

self.interpreter.moveToThread(self.thread)

but the objects you create in its constructor, i.e:

self.stream = ConsoleStream()

are not, as they have no parent associated with them.

daegontaven

@kshegunov Thank you so much for responding. I tried your method(setting a parent). It actually improved the response time significantly. Much less lag on windows machines now. But the blocking issue still persists. It would appear this issue has never been solved in all these years. So I've decided to abandon the QPlainTextEdit strategy. Is it possible to use QListView instead? How would I go about changing the code so it would work as close as possible to QPlainTextEdit?

SGaist

It's not a question of unsolved issue, QPlainTextEdit has never been designed to handle console like behaviour. AFAIK, what people do is to use a heavily styled QListView + QStyleItemDelegate combo with a custom model to allow more fine grained updates and avoid performance issue by using a rolling window over the data.

daegontaven

@SGaist Yes, that's what I meant to say. I wish it was built for faster processing of live data too. I think the issue with the signal from the buffer was that the emits are being queued in the event loop and unfortunately Python has a GIL which might be blocking the concurrent execution. So it might be an issue inherent to Python and not Qt per se.

As for the QListView, I think I understand what you're getting at when you say"rolling window". I did initially use QListView before ever starting with a buffer, but I saw that the performance was still bad. But now I think it was because like there is a setMaximumBlockCount() method for QPlainTextEdit, I never removed the items from the model. I was using QStringListModel and not a custom model. However, I am not familiar QStyledItemDelegate or custom models either. Could you direct me to some sources or examples that implement something like this? Thank you so much for the help.

Edit:
After some digging around, I found promising documentation for model view programming. I think this is what I was looking for. I want to thank everyone for sticking with my mess and helping me anyway. I'll mark this thread as closed. Thank you guys/gals once again.

daegontaven

Guys/Gals, I guess I gave up too quickly because someone on stack-overflow solved the problem. I'm just going to leave the solution here in case someone needs it in the future.

Here is the change that fixed the issue.

daegontaven

@SGaist @kshegunov I am afraid I have to change the status of this thread to unsolved because after successfully making a buffer that works with QPlainTextEdit (To remove delay in output). The problem started again because I set parents for every child of PythonInterpreter except for itself so I can move it to a thread when using the new buffer. Now the signals can't be emitted anymore again. It might not be a matter of them not emitting, but they are perhaps being queue in the EventLoop of the thread while InteractiveConsole.runcode is blocking.

When I say the signals can't be emitted, they can't be emitted when/if I run a blocking function like time.sleep() is run in the interpreter. So this subsequently makes InteractiveConsole.runcode blocking. Is there a way around this ?

Here is the new MCVE(Which is shorter now).

This is the signal not being emitted.

Any help would be amazing. Thank you !

SGaist

What would you want to happen while that blocking code runs ?

daegontaven

@SGaist I need the slot send_console_log to be called when the signal written in ConsoleStream is emitted. When running the program with a normal while/for loop in the interpreter, the slot will be called becuase at that moment its non blocking. But the moment I add time.sleep into a loop, slots will only be called at the end when all the blocking is over. Thank you for looking into this.