Issue with Worker Class when used with QTables

[imissthecommandline]

I'm having an issue in my program where utilizing threading is causing lag, but only if there is a QTableModel on the tab in which the Threads are started. I'm using the full signals + slots framework, and I'm only experiencing this lag when starting several functions from that specific tab of the program. Removing that specific QTableModel seems to fix the issue entirely. The lag also only exists if that QTableModel is on the same tab where I start the new threads, and only if I'm either on that tab, or switching to a tab with a QTableModel.

Is QTableModel only threadsafe if used once in a program?

[imissthecommandline]

ok so the workaround i discovered is initializing new threads from tabs without QTables. What confuses me is that you seem to be allowed at least one thread per tab with a Qtable before the issues start

[imissthecommandline]

the workaround does not in fact work. unclear what is going on at this point

[jsulm]

@imissthecommandline said in Issue with Worker Class when used with QTables:

Is QTableModel only threadsafe if used once in a program?

The documentation states that it is not: https://doc.qt.io/qt-6/qabstracttablemodel.html

"What confuses me is that you seem to be allowed at least one thread per tab" - not sure what you mean here. UI may only be accessed from main/UI thread!

[Pl45m4]

@imissthecommandline said in Issue with Worker Class when used with QTables:

unclear what is going on at this point

To make things clear, you could post what you did exactly and let others have a look :)
I smell bad design or unallowed/blocking UI access.

[imissthecommandline]

@Pl45m4

Here's all of the code, let me know what i messed up.

Sorry if it's messy, this is my first time with pyqt

# -*- coding: utf-8 -*-
"""
Created on Wed Jul 24 11:28:27 2024

@author: pierre
"""

#i tried to remove all the excess stuff unrelated to the core gui

#################
#Library Imports#
#################

#libraries for array management and graphing
import pandas as pd
import numpy as np
import matplotlib as plt

#libraries for system access and gui foundation
import sys 
from PyQt6.QtWidgets import (
    QApplication,
    QLabel,
    QMainWindow,
    QStatusBar,
    QToolBar,
    QStackedWidget,
    QStackedLayout,
    QWidget,
    QTabWidget,
    QVBoxLayout,
    QGridLayout,
    QPushButton,
    QLineEdit,
    QTableView
)

from PyQt6 import QtCore, QtGui, QtWidgets
from PyQt6.QtGui import *
from PyQt6.QtWidgets import *
from PyQt6.QtCore import *


#placeholder for stuff i haven't implemented in full yet
placeholder = "<unimplemented val!>"

#Library Imports for core management program
import socket
import threading
import time
import pickle


###############
#Global Values#
###############

#so i can edit the stuff in thw window class elsewhere
global window

#server necessities
server_ip = "0.0.0.0"
core_connection = 0
server_port = 9999
server_connections = 4
is_open = True

#list of collected devices
devices = []
#list of device states
device_states = []
#controller of current mode for all devices
current_mode = "waiting"
#generic method of encoding
code = "utf-8"
#list of unauthorized devices in network setup
unauthorized_devices = []
    
#stuff for demo
demo = []
avg = 0

#stuff for communication demo/tensorflow demo
iterations = 5
epochs = 2
model_type = "Convolutional Neural Network"
averaging_method = "All Layers"

#stuff for tensorflow
global weights
tensorflow_demo = []

#settings storage
settings = [f"{server_port}",f"{server_connections}",f"{iterations}",f"{epochs}",f"{model_type}",f"{averaging_method}"]

#################
#program classes#
#################

#code taken from pyqt tutorial (link below)
#https://www.pythonguis.com/tutorials/pyqt6-qtableview-modelviews-numpy-pandas/
class table_model(QtCore.QAbstractTableModel):
    def __init__(self, data):
        super(table_model, self).__init__()
        self._data = data

    def data(self, index, role):
        if role == Qt.ItemDataRole.DisplayRole:
            value = self._data.iloc[index.row(), index.column()]
            return str(value)

    def rowCount(self, index):
        return self._data.shape[0]

    def columnCount(self, index):
        return self._data.shape[1]

    def headerData(self, section, orientation, role):
        # section is the index of the column/row.
        if role == Qt.ItemDataRole.DisplayRole:
            if orientation == Qt.Orientation.Horizontal:
                return str(self._data.columns[section])

            if orientation == Qt.Orientation.Vertical:
                return str(self._data.index[section])
    
    def flags(self, index):
        return Qt.ItemFlag.ItemIsSelectable | Qt.ItemFlag.ItemIsEnabled | Qt.ItemFlag.ItemIsEditable
        
    #append the dataframe
    def appendSelf(self,new_val):
        self._data = pd.concat([self._data,new_val])
        self.layoutChanged.emit()
        return 0
        
    #edit a specific value
    def editSelf(self,new_val,index,column):
        self._data.at[index,column] = new_val
        self.layoutChanged.emit()
        return 0
    
    #remove a line
    def removeSelf(self,index):
        self._data.set_index(index)
        self._data.reset_index(drop=True)
        self.layoutChanged.emit()
        return 0
        

#Main GUI window coding
class Window(QMainWindow):
    def __init__(self):
        super().__init__(parent=None)
        #values for window resolution
        self.x_res = 640
        self.y_res = 480
        self.setWindowTitle("S.T.A.R.F.I.S.H")
        #various set-up functions for the gui
        self.menu_init()
        self.statusbar_init() 
        self.stack_init()
        self.setGeometry(0, 30, self.x_res, self.y_res)
        #setup for thread manager
        self.threadpool = QThreadPool()

    def menu_init(self):
        #setup generic options menu for window 
        menu = self.menuBar().addMenu("&Menu")
        menu.addAction("&Exit", self.close)
        
        #initialization for the resolutions menu
        #lambda is used to circumvent constraints of addAction
        #functions with arguments are otherwise deemed heretics and burned
        res_menu = menu.addMenu("&Resolutions")
        res_1 = res_menu.addAction("640 by 480")
        res_1.triggered.connect(lambda: self.set_display("Nan", 640, 480))
        res_2 = res_menu.addAction("1024 by 768")
        res_2.triggered.connect(lambda: self.set_display("Nan", 1024, 768))
        res_3 = res_menu.addAction("1280 by 720")
        res_3.triggered.connect(lambda: self.set_display("Nan",1280, 720))
        res_3 = res_menu.addAction("1366 by 768")
        res_3.triggered.connect(lambda: self.set_display("Nan",1366, 768))
        res_4 = res_menu.addAction("1920 by 1080")
        res_4.triggered.connect(lambda: self.set_display("Nan",1920, 1080))
        res_5 = res_menu.addAction("Fullscreen")
        res_5.triggered.connect(lambda: self.set_display("fullscreen"))

    def statusbar_init(self):
        status = QStatusBar()
        status.showMessage("Currently Running: Nothing currently running...")
        self.setStatusBar(status)
        
    def stack_init(self):
        #all possible screens are organized through tabs at the top
        #all sub-tabs are part of a stack
        
        #basic setup
        self.layout = QVBoxLayout(self)
        self.main_tabs = QTabWidget(self)
        self.main_tabs.resize(self.x_res - 70, self.y_res - 70) 
        self.main_tabs.move(10, 40) 
        
        #custom tab init
        self.home_tab = QWidget(self)
        self.main_tabs.addTab(self.home_tab,"Home")
        self.config_tab = QWidget(self)
        self.main_tabs.addTab(self.config_tab,"Configuration")
        self.conn_tab = QWidget(self)
        self.main_tabs.addTab(self.conn_tab,"Connections")
        self.conn_man_tab = QWidget(self)
        self.main_tabs.addTab(self.conn_man_tab,"Connection Management")
        self.run_tab = QWidget(self)
        self.main_tabs.addTab(self.run_tab,"Run")
        self.layout.addWidget(self.main_tabs) 
        self.setLayout(self.layout) 
        
        #home tab setup
        #label 1 formatting
        self.home_tab.layout = QVBoxLayout(self)
        self.top_label = QLabel()
        self.top_label.setText("Welcome to S.T.A.R.F.I.S.H") 
        self.home_tab.layout.addWidget(self.top_label)
        #label 2 formatting
        self.starfish_image = QLabel()
        self.starfish_image.setPixmap(QPixmap('starfishe.png'))
        self.home_tab.layout.addWidget(self.starfish_image)
        #label 3 formatting
        self.blurb = QLabel("To upload a config file, or to manually adjust settings, go to Configuration \n\nTo connect new devices to the server, or to check on the state of existing connections, go to Connections\n\nTo run a model of your choice, and to see the status of the current overall model, go to Run", self)
        self.blurb.setWordWrap(True)
        self.blurb.setStyleSheet("border: 2px solid blue;") 
        self.home_tab.layout.addWidget(self.blurb)
        self.home_tab.setLayout(self.home_tab.layout)
        
        #options tab setup
        #each adjustable setting has a qlabel in column 0 saying what it is
        #this what the value is and the current value
        #a input box in column 4 lets you change the value
        
        self.config_tab.layout = QGridLayout(self)
        self.upload_button = QPushButton("&Upload custom config", self)
        self.config_tab.layout.addWidget(self.upload_button,0,0,1,5)
        #columns 1,3,5 are thin for l'aesthétique :)
        for i in range(3):
            self.config_tab.layout.setColumnMinimumWidth(1 + (i*2), 10)
            
        #network settings start
        #need to implement, not bothering yet because it's not really necessary
        self.config_ops_1 = QLabel("Network settings", self)
        self.config_ops_1.setStyleSheet("border: 2px solid blue;") 
        self.config_tab.layout.addWidget(self.config_ops_1,1,0)
        self.config_ops_note = QLabel("To change the value, use the adjacent input box!", self)
        self.config_tab.layout.addWidget(self.config_ops_note,1,2)
        self.network_op_1_1 = QLabel(f"Port: {settings[0]}", self)
        self.config_tab.layout.addWidget(self.network_op_1_1,2,0)
        self.network_op_1_2 = QLineEdit(self)
        self.config_tab.layout.addWidget(self.network_op_1_2,2,2)
        self.network_op_2_1 = QLabel(f"Number of Devices: {settings[1]}", self)
        self.config_tab.layout.addWidget(self.network_op_2_1,3,0)
        self.network_op_2_2 = QLineEdit(self)
        self.config_tab.layout.addWidget(self.network_op_2_2,3,2)
        #tensorflow demo settings start
        self.config_ops_2 = QLabel("Tensorflow Demo Settings", self)
        self.config_ops_2.setStyleSheet("border: 2px solid green;") 
        self.config_tab.layout.addWidget(self.config_ops_2,4,0)
        self.tensor_demo_op_1_1 = QLabel(f"Number of Iterations: {settings[2]}", self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_1_1,5,0)
        self.tensor_demo_op_1_2 = QLineEdit(self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_1_2,5,2,1,1)
        self.tensor_demo_op_2_1 = QLabel(f"Number of Epochs: {settings[3]}", self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_2_1,6,0)
        self.tensor_demo_op_2_2 = QLineEdit(self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_2_2,6,2,1,1)
        self.tensor_demo_op_3_1 = QLabel(f"Model Type: {settings[4]}", self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_3_1,7,0)
        self.tensor_demo_op_3_2 = QLineEdit(self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_3_2,7,2,1,1)
        self.tensor_demo_op_4_1 = QLabel(f"Specific Layer Avg: {settings[5]}", self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_4_1,8,0)
        self.tensor_demo_op_4_2 = QLineEdit(self)
        self.config_tab.layout.addWidget(self.tensor_demo_op_4_2,8,2,1,1)
        self.config_tab.setLayout(self.config_tab.layout)
        
        #connections tab setup
        #if nothing is connected, top button starts listener
        #if listener is running, bottom bar says as such
        #every time a device is connected, it is added to the list of devices
        #once listener is done, button at the top disconnects devices instead
        
        self.conn_tab.layout = QGridLayout(self)
        self.start_connecting = QPushButton("&Open Server for Connections", self)
        self.start_connecting.clicked.connect(lambda: self.threadstarter(spinup))
        self.conn_tab.layout.addWidget(self.start_connecting,0,0,1,2)
        self.listener_running = QLabel("Listener is runnning...", self)
        self.conn_tab.layout.addWidget(self.listener_running,0,0,1,2)
        self.listener_running.hide()
        self.start_connecting = QPushButton("&Close Server", self)
        self.start_connecting.clicked.connect(lambda: self.close_server)
        self.conn_tab.layout.addWidget(self.start_connecting,0,2,1,2)
        #columns 1,3,5 are thin for l'aesthétique :)
        for i in range(3):
            self.config_tab.layout.setColumnMinimumWidth(1 + (i*2), 10)
        self.device_list_start = QLabel("Connected Devices",self)
        self.device_list_start.setStyleSheet("border: 2px solid blue;")
        self.conn_tab.layout.addWidget(self.device_list_start,1,0,1,1)
        #initially array for connected devices
        self.conn_devices_table = QTableView()
        self.connected_devices = pd.DataFrame([
            [f"{server_ip}", f"{server_port}", "Connections Tab",f"{current_mode}"],
            ],columns = ["Ipv4 Address", "Port", "Device State","Current Program"],
            index = ["Server"]
            )
        self.device_list = table_model(self.connected_devices)
        self.conn_devices_table.setModel(self.device_list)
        self.conn_tab.layout.addWidget(self.conn_devices_table,2,0,5,5)
        self.conn_tab.setLayout(self.conn_tab.layout)
        
        #connection management tab setup
        
        #button for authorizing all connections, button for clearing network
        self.conn_man_tab.layout = QGridLayout(self)
        self.authorize_connections = QPushButton("&Authorize all Connections", self)
        self.authorize_connections.clicked.connect(lambda: self.connection_authorizer())
        self.conn_man_tab.layout.addWidget(self.authorize_connections,0,0,1,2)
        self.clear_network_button = QPushButton("&Clear Network", self)
        self.clear_network_button.clicked.connect(lambda: self.clear_server())
        self.conn_man_tab.layout.addWidget(self.clear_network_button,0,2,1,2)
        self.conn_man_tab.setLayout(self.conn_man_tab.layout)
        
        #run tab setup
        self.run_tab.layout = QGridLayout(self)
        self.run_device_label = QLabel("Implemented Models",self)
        self.run_device_label.setStyleSheet("border: 2px solid blue;")
        self.run_tab.layout.addWidget(self.run_device_label,0,0,1,1)
        self.tensorflow_demo_button = QPushButton("&Run Basic Tensorflow Demo", self)
        self.tensorflow_demo_button.clicked.connect(lambda: self.threadstarter(tensorflow_basic_demo))
        self.run_tab.layout.addWidget(self.tensorflow_demo_button,1,0,1,1)
        self.run_device_label = QLabel("Connected Devices",self)
        self.run_device_label.setStyleSheet("border: 2px solid green;")
        self.run_tab.layout.addWidget(self.run_device_label,3,0,1,1)
        self.run_device_table = QTableView()
        self.connected_devices_run = pd.DataFrame([
            ["Running",f"{current_mode}","N/A"],
            ],columns = ["Device State","Current Program","Last Ping"],
            index = ["Server"]
        )
        self.device_list_run = table_model(self.connected_devices_run)
        self.run_device_table.setModel(self.device_list_run)
        self.run_tab.layout.addWidget(self.run_device_table,4,0,5,5)
        self.run_tab.setLayout(self.run_tab.layout)
        
    def set_display(self, fit="Nan", x_res = 640, y_res = 480):
        if (fit == "fullscreen"):
            self.showMaximized() 
            self.main_tabs.resize(x_res - 80, y_res - 80) 
        else:
            self.setGeometry(0, 30, x_res, y_res)
            self.main_tabs.resize(x_res - 80, y_res - 80) 
    
    def threadstarter(self, function, *args):
        new_thread = Worker(function, *args)
        new_thread.signals.operations.connect(self.threadhandler)
        self.threadpool.start(new_thread)
        
    #note for any additions, is set to expect a tuple
    #sending an int closes the thread, unless the int is in a tuple (duh)
    def threadhandler(self, command_list):
        task = command_list[0]
        if (task[0] == "val_edit_conn"):
            self.device_list.editSelf(task[1],task[2],task[3])
        elif(task[0] == "val_edit_run"):
            self.device_list_run.editSelf(task[1],task[2],task[3])
        elif(task[0] == "append_conn"):
            pd_array = command_list[1]
            self.device_list.appendSelf(pd_array)
        elif(task[0] == "append_run"):
            pd_array = command_list[1]
            self.device_list_run.appendSelf(pd_array)
        elif(task[0] == "server_status"):
            self.server_mode_update()
        elif(task[0] == "set_status"):
            status = QStatusBar()
            status.showMessage(f"Currently Running: {task[1]}")
            self.setStatusBar(status)
        elif (task[0] == "tray_start"):
            self.threadstarter(arbitrary_function,command_list[1],command_list[2])
            
    def update_status(self,new_status):
        status = QStatusBar()
        status.showMessage(new_status)
        self.setStatusBar(status)
        pass
    
    def server_mode_update(self):
        self.device_list_run.editSelf(f"{current_mode}","Server","Current Program")
        current_time = time.localtime()
        current_time = time.strftime("%H:%M:%S", current_time)
        self.device_list_run.editSelf(current_time,"Server","Last Ping")
        self.device_list.editSelf(f"{current_mode}","Server","Current Program")
        
    #used as a workaround for performance issues with threads started by threads
    def connection_authorizer(self):
        global unauthorized_devices
        if (unauthorized_devices == []):
            return 0
        for unauthorized_device in unauthorized_devices:
            current_device = unauthorized_device
            self.threadstarter(arbitrary_function,current_device[0],current_device[1])
        unauthorized_devices = []
        return 0
    
    def close_server(self):
        global is_open
        is_open = False
    
    def clear_server(self):
        global current_mode
        current_mode = "newtwork_wipe"
        
    def update_settings(self):
        #why can't things just be command line
        #: (
        pass
            
            
class connection:
    def __init__(self, ip, conn, name, port=9999):
      self.ip = ip
      self.conn = conn
      self.name = name
      self.port = port
      
    #function for making sending strings slightly easier
    def send(self,message):
        #added to prevent improperly formatted messages from throwing errors
        message = str(message)
        message_size = str(len(message))
        self.conn.send(message_size.encode(code))
        #delay to prevent signal interference
        time.sleep(0.5)
        self.conn.send(message.encode(code))
        
    #function for recieving special datatypes
    #may add full file sending here, who knows
    def send_advanced(self,message,datatype="array"):
        if datatype == "array":
            #added to prevent improperly formatted messages from throwing errors
            message = pickle.dumps(message)
            message_size = str(len(message))
            #print(message_size)
            self.conn.send(message_size.encode(code))
            #delay to prevent signal interference
            time.sleep(0.5)
            self.conn.send(message)
            
    #function for making recieving strings slightly easier
    def recieve(self):
        message_size = int(self.conn.recv(64).decode(code))
        #delay to prevent signal interference
        time.sleep(0.5)
        message = self.conn.recv(message_size).decode(code)
        return message
    
    #function for recieving special datatypes
    #may accept files in the future
    def recieve_advanced(self):
        message_size = int(self.conn.recv(64).decode(code))
        #delay to prevent signal interference
        time.sleep(0.5)
        message = self.conn.recv(message_size)
        message = pickle.loads(message)
        return message
    
       
#allows for utilization of threads in gui backend
class Worker(QRunnable):
    def __init__(self, fn, *args):
        super(Worker, self).__init__()
        # Store constructor arguments (re-used for processing)
        self.fn = fn
        self.args = args
        self.signals = WorkerSignals()

    @pyqtSlot()
    def run(self):
        '''
        Initialise the runner function with passed args, kwargs.
        '''
        self.fn(self, *self.args)
        
#implementation of Qobject for handling signals sent from worker threads
class WorkerSignals(QObject):
    operations = pyqtSignal(tuple)
    
    
    
###################
#program functions#
###################

#print statements still exist for debuggings sake

#it's a function like this because it's a thread
def spinup(thread_manager):
    thread_manager.signals.operations.emit((["set_status","Connecting Devices..."],))
    client = setup(thread_manager)
    listener(client,thread_manager)
    thread_manager.signals.operations.emit((["set_status","Nothing Currently Running..."],))
    return 0

#note to self: this needs to change to settings at some point
def setup(thread_manager,port=server_port,connections=server_connections):
    global server_ip
    global core_connetion
    global current_mode
    current_mode = "waiting"
    client = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
    ip = socket.gethostbyname(socket.gethostname())
    server_ip = ip
    thread_manager.signals.operations.emit((["val_edit_conn", f"{server_ip}", "Server", "Ipv4 Address"],))
    core_connection = client
    client.bind((ip,port))
    client.listen(connections)
    #print("Server set up is complete")
    #print(f"Server ip is {ip}")
    #print(f"Server port is {port}")
    #print(f"Server allows up to {connections} devices")
    return client

def listener(client,thread_manager):
    global window
    global unauthorized_devices
    global is_open
    client.listen()
    dev_count = 0
    pi4_count = 0
    arduino_uno_count = 0
    is_open = True
    while (is_open):
        if (dev_count<4):
            print(f"\nListener is available for new connections {dev_count} connected\n")
            dev_count = dev_count + 1
        else:
            break
        conn, addr = client.accept()
        name = f"pi-{dev_count}"
        new_connection = connection(addr, conn, name)
        new_connection.send("type")
        dev_type = new_connection.recieve()
        if (dev_type == "pi4"):
            pi4_count = pi4_count + 1
            new_connection.name = f"pi4-{pi4_count}"
        elif (dev_type == "arduino uno"):
            arduino_uno_count = arduino_uno_count + 1
            new_connection.name = f"arduino_uno-{arduino_uno_count}"
        name = new_connection.name
        #thread_manager.signals.operations.emit((["tray_start"],new_connection,dev_count))
        unauthorized_devices.append((new_connection,dev_count))
        devices.append(new_connection)
        device_states.append("awaiting authorization...")
        demo.append(0)
        tensorflow_demo.append(0)
        current_time = time.localtime()
        current_time = time.strftime("%H:%M:%S", current_time)
        new_device = pd.DataFrame([
            [f"{addr[0]}", f"{server_port}", f"{device_states[dev_count - 1]}",f"{current_mode}",f"{current_time}"],
            ],columns = ['Ipv4 Address', "Port", "Device State","Current Program","Last Ping"],
            index = [f'{name}']
        )
        thread_manager.signals.operations.emit((["append_conn"],new_device))
        new_device = pd.DataFrame([
            [f"{device_states[dev_count - 1]}",f"{current_mode}",f"{current_time}"],
            ],columns = ["Device State","Current Program","Last Ping"],
            index = [f'{name}']
        )
        thread_manager.signals.operations.emit((["append_run"],new_device))
    print("Max device count reached, server closing...")
    thread_manager.signals.operations.emit((["server_status"],))
    return 0
        #print(f"pi-{pi_count} is waiting for instructions...")

#function for ensuring device consensus accross the network
def consensus(target_list,goal):
    #waits until at least one device reaches the state we want
    while(target_list[0] != goal):
        pass
    #waits until all devices read the same state
    while(target_list.count(target_list[0]) != len(target_list)):
        pass
    
#resets global variables between mode executions
def reset():
    global current_mode
    global avg
    global demo
    global iterations
    global tensorflow_demo
    for i in range(len(demo)):
        demo[i] = 0
    for i in range(len(tensorflow_demo)):
        tensorflow_demo[i] = 0
    avg = 0
    iterations = 0
    current_mode = "waiting"
    
#arbitraty function for testing
def arbitrary_function(thread_manager, *stuff):
    while(True):
        pass


#handles all changing of state *insert pun here*
def state_changer(thread_manager,pi_count,name,new_state):
    device_states[pi_count-1] = new_state
    test = ["val_edit_run",f"{new_state}",f"{name}"]
    thread_manager.signals.operations.emit((["val_edit_run",f"{new_state}",f"{name}", "Device State"],))
    thread_manager.signals.operations.emit((["val_edit_conn",f"{current_mode}",f"{name}", "Current Program"],))
    thread_manager.signals.operations.emit((["val_edit_run",f"{current_mode}",f"{name}", "Current Program"],))
    current_time = time.localtime()
    current_time = time.strftime("%H:%M:%S", current_time)
    thread_manager.signals.operations.emit((["val_edit_conn",f"{current_time}",f"{name}", "Last Ping"],))
    
def tensorflow_basic_demo(thread_manager):
    #not really related to the problem
    pass
###########
#Main Loop#
###########

def maine():
    global window
    app = QApplication([])
    window = Window()
    window.show()
    
    sys.exit(app.exec())
    
#call main to start program
maine()

[Pl45m4]

@imissthecommandline

Urgh, PyQt :)
You should have mentioned that before or post that in Qt for Python category.
Since I can't spot the issue from the first glance and I'm currently not able to run your code, I can't help you.

[imissthecommandline]

got it, posting over there

thank you for looking!

[Pl45m4]

@imissthecommandline

Don't need to post it again.
Some mod (@moderators) might move this post there.

[imissthecommandline]

got it, thank you

[SGaist]

Hi,

This is more than 600 lines of pretty convoluted code that uses way too many globals.
If you want an answer please reduce it so that it can be used to reproduce your issue.

[imissthecommandline]

@SGaist
ouch, sorry about that

here's a version with pretty much all but the essentials removed:

# -*- coding: utf-8 -*-
"""
Created on Wed Jul 24 15:22:55 2024

@author: pierre
"""

#libraries for array management and graphing
import pandas as pd
import numpy as np
import matplotlib as plt

#libraries for system access and gui foundation
import sys 
from PyQt6.QtWidgets import (
    QApplication,
    QLabel,
    QMainWindow,
    QStatusBar,
    QToolBar,
    QStackedWidget,
    QStackedLayout,
    QWidget,
    QTabWidget,
    QVBoxLayout,
    QGridLayout,
    QPushButton,
    QLineEdit,
    QTableView
)

from PyQt6 import QtCore, QtGui, QtWidgets
from PyQt6.QtGui import *
from PyQt6.QtWidgets import *
from PyQt6.QtCore import *


#placeholde for stuff i haven't implemented in full yet
placeholder = "<unimplemented val!>"

#Library Imports for core management program
import socket
import threading
import time
import pickle

global window

#allows for utilization of threads in gui backend
class Worker(QRunnable):
    def __init__(self, fn, *args):
        super(Worker, self).__init__()
        # Store constructor arguments (re-used for processing)
        self.fn = fn
        self.args = args
        self.signals = WorkerSignals()

    @pyqtSlot()
    def run(self):
        '''
        Initialise the runner function with passed args, kwargs.
        '''
        self.fn(self, *self.args)
        
#implementation of Qobject for handling signals sent from worker threads
class WorkerSignals(QObject):
    operations = pyqtSignal(tuple)
    

#code taken from pyqt tutorial (link below)
#https://www.pythonguis.com/tutorials/pyqt6-qtableview-modelviews-numpy-pandas/
class table_model(QtCore.QAbstractTableModel):
    def __init__(self, data):
        super(table_model, self).__init__()
        self._data = data

    def data(self, index, role):
        if role == Qt.ItemDataRole.DisplayRole:
            value = self._data.iloc[index.row(), index.column()]
            return str(value)

    def rowCount(self, index):
        return self._data.shape[0]

    def columnCount(self, index):
        return self._data.shape[1]

    def headerData(self, section, orientation, role):
        # section is the index of the column/row.
        if role == Qt.ItemDataRole.DisplayRole:
            if orientation == Qt.Orientation.Horizontal:
                return str(self._data.columns[section])

            if orientation == Qt.Orientation.Vertical:
                return str(self._data.index[section])
    
    def flags(self, index):
        return Qt.ItemFlag.ItemIsSelectable | Qt.ItemFlag.ItemIsEnabled | Qt.ItemFlag.ItemIsEditable
        
    #append the dataframe
    def appendSelf(self,new_val):
        self._data = pd.concat([self._data,new_val])
        self.layoutChanged.emit()
        return 0
        
    #edit a specific value
    def editSelf(self,new_val,index,column):
        self._data.at[index,column] = new_val
        self.layoutChanged.emit()
        return 0
    
    #remove a line
    def removeSelf(self,index):
        self._data.set_index(index)
        self._data.reset_index(drop=True)
        self.layoutChanged.emit()
        return 0
    
#Main GUI window coding
class Window(QMainWindow):
    def __init__(self):
        super().__init__(parent=None)
        #values for window resolution
        self.x_res = 640
        self.y_res = 480
        self.setWindowTitle("S.T.A.R.F.I.S.H")
        #various set-up functions for the gui
        self.stack_init()
        self.setGeometry(0, 30, self.x_res, self.y_res)
        #setup for thread manager
        self.threadpool = QThreadPool()

        
    def stack_init(self):
        #all possible screens are organized through tabs at the top
        #all sub-tabs are part of a stack
        
        #basic setup
        self.layout = QVBoxLayout(self)
        self.main_tabs = QTabWidget(self)
        self.main_tabs.resize(self.x_res - 70, self.y_res - 70) 
        self.main_tabs.move(10, 40) 
        
        #custom tab init
        self.home_tab = QWidget(self)
        self.main_tabs.addTab(self.home_tab,"Home")
        self.config_tab = QWidget(self)
        self.main_tabs.addTab(self.config_tab,"Configuration")
        self.conn_tab = QWidget(self)
        self.main_tabs.addTab(self.conn_tab,"Connections")
        self.conn_man_tab = QWidget(self)
        self.main_tabs.addTab(self.conn_man_tab,"Connection Management")
        self.run_tab = QWidget(self)
        self.main_tabs.addTab(self.run_tab,"Run")
        self.layout.addWidget(self.main_tabs) 
        self.setLayout(self.layout) 
        
        #home tab setup
        #label 1 formatting
        self.home_tab.layout = QVBoxLayout(self)
        self.top_label = QLabel()
        self.top_label.setText("BlaBlaBla this tab isn't the problem") 
        self.home_tab.layout.addWidget(self.top_label)

        self.home_tab.setLayout(self.home_tab.layout)
        
        #options tab setup
        #each adjustable setting has a qlabel in column 0 saying what it is
        #this what the value is and the current value
        #a input box in column 4 lets you change the value
        
        self.config_tab.layout = QGridLayout(self)
        self.upload_button = QPushButton("&Upload custom config", self)
        self.config_tab.layout.addWidget(self.upload_button,0,0,1,5)
        #columns 1,3,5 are thin for l'aesthétique :)
        for i in range(3):
            self.config_tab.layout.setColumnMinimumWidth(1 + (i*2), 10)
            
        #network settings start
        self.config_ops_1 = QLabel("Go to the connections tab", self)
        self.config_ops_1.setStyleSheet("border: 2px solid blue;") 
        self.config_tab.layout.addWidget(self.config_ops_1,1,0)
        
        self.config_tab.setLayout(self.config_tab.layout)
        
        #connections tab setup

        
        self.conn_tab.layout = QGridLayout(self)
        self.start_connecting = QPushButton("&Start a thread", self)
        self.start_connecting.clicked.connect(lambda: self.threadstarter(arbitrary_task))
        self.conn_tab.layout.addWidget(self.start_connecting,0,0,1,2)
        #initially array for connected devices
        self.conn_devices_table = QTableView()
        self.connected_devices = pd.DataFrame([
            ["{server_ip}", "{server_port}", "Connections Tab","{current_mode}"],
            ],columns = ["Ipv4 Address", "Port", "Device State","Current Program"],
            index = ["Server"]
            )
        self.device_list = table_model(self.connected_devices)
        self.conn_devices_table.setModel(self.device_list)
        self.conn_tab.layout.addWidget(self.conn_devices_table,2,0,5,5)
        self.conn_tab.setLayout(self.conn_tab.layout)
        
        #connection management tab setup
        
        #button for authorizing all connections, button for clearing network
        self.conn_man_tab.layout = QGridLayout(self)
        self.authorize_connections = QPushButton("&Start a thread", self)
        self.authorize_connections.clicked.connect(lambda: self.connection_authorizer(arbitrary_task))
        self.conn_man_tab.layout.addWidget(self.authorize_connections,0,0,1,2)
        self.conn_man_tab.setLayout(self.conn_man_tab.layout)
        
        #run tab setup
        self.run_tab.layout = QGridLayout(self)
        self.run_device_label = QLabel("Other Table I Guess",self)
        self.run_device_label.setStyleSheet("border: 2px solid blue;")
        self.run_tab.layout.addWidget(self.run_device_label,0,0,1,1)
        
        self.run_device_table = QTableView()
        self.connected_devices_run = pd.DataFrame([
            ["Running","{current_mode}","N/A"],
            ],columns = ["Device State","Current Program","Last Ping"],
            index = ["Server"]
        )
        self.device_list_run = table_model(self.connected_devices_run)
        self.run_device_table.setModel(self.device_list_run)
        self.run_tab.layout.addWidget(self.run_device_table,4,0,5,5)
        
        self.run_tab.setLayout(self.run_tab.layout)
    
    def threadstarter(self, function, *args):
        new_thread = Worker(function, *args)
        new_thread.signals.operations.connect(self.threadhandler)
        self.threadpool.start(new_thread)
        
    #note for any additions, is set to expect a tuple
    #sending an int closes the thread, unless the int is in a tuple (duh)
    def threadhandler(self, command_list):
        pass

def arbitrary_task(*args):
    print("Started Thread")
    while(True):
        pass

def main():
    global window
    app = QApplication([])
    window = Window()
    window.show()
    sys.exit(app.exec())
    
#call main to start program
main()

it seems to only lag on tabs with tables, even though we aren't interacting with them. any ideas as to why this is?

[SGaist]

What is the procedure to reproduce your issue with that version ?
By the way it crashes on line 210 when clicking "Start a thread" on the "Connection management" tab.

[imissthecommandline]

@SGaist the connected function on line 210 should be

lambda: self.threadstarter(arbitrary_task))

I messed up switching it out for the simplified version.

If you click the start thread button a bunch of times, the program lags, but only when going to a tab with a table or from a tab with a table removing the connections tab table removes the lag.

To reproduce:
Run Program
Click on Connections Tab
Click the start thread button 8 or so times
Click around the tabs

I think pyqt6 might have a hard time managing threads and tables, but it's probably my implementation, as this is my first time with pyqt6

Thank you

[imissthecommandline]

Ok so i've figured out some interesting things about this:

1. Using the .hide() function on the table removes the lag. using .hide() on both of the tables before starting a new thread then .show() after the thread is started moderately reduces the lag.
2. Completely reinitializing the table by removing the widget from the layout then adding it again after starting a new thread reduces the lag more than method 1, but there's still some noticeable lag
   3.Adding the tables AFTER starting the threads has no impact at all, the lag is just as bad

Still not sure what the issue is though

Here's the code for anyone insane enough to try and help

# -*- coding: utf-8 -*-
"""
Created on Wed Jul 24 15:22:55 2024

@author: pierre
"""

#libraries for array management and graphing
import pandas as pd
import numpy as np
import matplotlib as plt

#libraries for system access and gui foundation
import sys 
from PyQt6.QtWidgets import (
    QApplication,
    QLabel,
    QMainWindow,
    QStatusBar,
    QToolBar,
    QStackedWidget,
    QStackedLayout,
    QWidget,
    QTabWidget,
    QVBoxLayout,
    QGridLayout,
    QPushButton,
    QLineEdit,
    QTableView
)

from PyQt6 import QtCore, QtGui, QtWidgets
from PyQt6.QtGui import *
from PyQt6.QtWidgets import *
from PyQt6.QtCore import *


#placeholde for stuff i haven't implemented in full yet
placeholder = "<unimplemented val!>"

#Library Imports for core management program
import socket
import threading
import time
import pickle

global window

#allows for utilization of threads in gui backend
class Worker(QRunnable):
    def __init__(self, fn, *args):
        super(Worker, self).__init__()
        # Store constructor arguments (re-used for processing)
        self.fn = fn
        self.args = args
        self.signals = WorkerSignals()

    @pyqtSlot()
    def run(self):
        '''
        Initialise the runner function with passed args, kwargs.
        '''
        self.fn(self, *self.args)
        
#implementation of Qobject for handling signals sent from worker threads
class WorkerSignals(QObject):
    operations = pyqtSignal(tuple)
    

#code taken from pyqt tutorial (link below)
#https://www.pythonguis.com/tutorials/pyqt6-qtableview-modelviews-numpy-pandas/
class table_model(QtCore.QAbstractTableModel):
    def __init__(self, data):
        super(table_model, self).__init__()
        self._data = data

    def data(self, index, role):
        if role == Qt.ItemDataRole.DisplayRole:
            value = self._data.iloc[index.row(), index.column()]
            return str(value)

    def rowCount(self, index):
        return self._data.shape[0]

    def columnCount(self, index):
        return self._data.shape[1]

    def headerData(self, section, orientation, role):
        # section is the index of the column/row.
        if role == Qt.ItemDataRole.DisplayRole:
            if orientation == Qt.Orientation.Horizontal:
                return str(self._data.columns[section])

            if orientation == Qt.Orientation.Vertical:
                return str(self._data.index[section])
    
    def flags(self, index):
        return Qt.ItemFlag.ItemIsSelectable | Qt.ItemFlag.ItemIsEnabled | Qt.ItemFlag.ItemIsEditable
        
    #append the dataframe
    def appendSelf(self,new_val):
        self._data = pd.concat([self._data,new_val])
        self.layoutChanged.emit()
        return 0
        
    #edit a specific value
    def editSelf(self,new_val,index,column):
        self._data.at[index,column] = new_val
        self.layoutChanged.emit()
        return 0
    
    #remove a line
    def removeSelf(self,index):
        self._data.set_index(index)
        self._data.reset_index(drop=True)
        self.layoutChanged.emit()
        return 0
    
#Main GUI window coding
class Window(QMainWindow):
    def __init__(self):
        super().__init__(parent=None)
        #values for window resolution
        self.x_res = 640
        self.y_res = 480
        self.setWindowTitle("S.T.A.R.F.I.S.H")
        #various set-up functions for the gui
        self.stack_init()
        self.setGeometry(0, 30, self.x_res, self.y_res)
        #setup for thread manager
        self.threadpool = QThreadPool()

        
    def stack_init(self):
        #all possible screens are organized through tabs at the top
        #all sub-tabs are part of a stack
        
        #basic setup
        self.layout = QVBoxLayout(self)
        self.main_tabs = QTabWidget(self)
        self.main_tabs.resize(self.x_res - 70, self.y_res - 70) 
        self.main_tabs.move(10, 40) 
        
        #custom tab init
        self.home_tab = QWidget(self)
        self.main_tabs.addTab(self.home_tab,"Home")
        self.config_tab = QWidget(self)
        self.main_tabs.addTab(self.config_tab,"Configuration")
        self.conn_tab = QWidget(self)
        self.main_tabs.addTab(self.conn_tab,"Connections")
        self.conn_man_tab = QWidget(self)
        self.main_tabs.addTab(self.conn_man_tab,"Connection Management")
        self.run_tab = QWidget(self)
        self.main_tabs.addTab(self.run_tab,"Run")
        self.layout.addWidget(self.main_tabs) 
        self.setLayout(self.layout) 
        
        #home tab setup
        #label 1 formatting
        self.home_tab.layout = QVBoxLayout(self)
        self.top_label = QLabel()
        self.top_label.setText("BlaBlaBla this tab isn't the problem") 
        self.home_tab.layout.addWidget(self.top_label)

        self.home_tab.setLayout(self.home_tab.layout)
        
        #options tab setup
        #each adjustable setting has a qlabel in column 0 saying what it is
        #this what the value is and the current value
        #a input box in column 4 lets you change the value
        
        self.config_tab.layout = QGridLayout(self)
        self.upload_button = QPushButton("&Upload custom config", self)
        self.config_tab.layout.addWidget(self.upload_button,0,0,1,5)
        #columns 1,3,5 are thin for l'aesthétique :)
        for i in range(3):
            self.config_tab.layout.setColumnMinimumWidth(1 + (i*2), 10)
            
        #network settings start
        self.config_ops_1 = QLabel("Go to the connections tab", self)
        self.config_ops_1.setStyleSheet("border: 2px solid blue;") 
        self.config_tab.layout.addWidget(self.config_ops_1,1,0)
        
        self.config_tab.setLayout(self.config_tab.layout)
        
        #connections tab setup

        
        self.conn_tab.layout = QGridLayout(self)
        self.start_connecting = QPushButton("&Start a thread", self)
        self.start_connecting.clicked.connect(lambda: self.threadstarter(arbitrary_task))
        self.conn_tab.layout.addWidget(self.start_connecting,0,0,1,2)
        #initial array for connected devices
        self.conn_devices_table = QTableView()
        self.connected_devices = pd.DataFrame([
            ["{server_ip}", "{server_port}", "Connections Tab","{current_mode}"],
            ],columns = ["Ipv4 Address", "Port", "Device State","Current Program"],
            index = ["Server"]
            )
        self.device_list = table_model(self.connected_devices)
        self.conn_devices_table.setModel(self.device_list)
        self.conn_tab.layout.addWidget(self.conn_devices_table,2,0,5,5)
        self.conn_tab.setLayout(self.conn_tab.layout)
        
        #connection management tab setup
        
        #button for authorizing all connections, button for clearing network
        self.conn_man_tab.layout = QGridLayout(self)
        self.authorize_connections = QPushButton("&Tabletime", self)
        self.authorize_connections.clicked.connect(lambda: self.testingthing())
        self.conn_man_tab.layout.addWidget(self.authorize_connections,0,0,1,2)
        self.conn_man_tab.setLayout(self.conn_man_tab.layout)
        
        #run tab setup
        self.run_tab.layout = QGridLayout(self)
        self.run_device_label = QLabel("Other Table I Guess",self)
        self.run_device_label.setStyleSheet("border: 2px solid blue;")
        self.run_tab.layout.addWidget(self.run_device_label,0,0,1,1)
        
        self.run_device_table = QTableView()
        self.connected_devices_run = pd.DataFrame([
            ["Running","{current_mode}","N/A"],
            ],columns = ["Device State","Current Program","Last Ping"],
            index = ["Server"]
        )
        self.device_list_run = table_model(self.connected_devices_run)
        self.run_device_table.setModel(self.device_list_run)
        self.run_tab.layout.addWidget(self.run_device_table,4,0,5,5)
        self.run_tab.setLayout(self.run_tab.layout)
    
    def threadstarter(self, function, *args):
        #the mechanics of this frighten me
        self.conn_tab.layout.removeWidget(self.conn_devices_table)
        self.conn_tab.setLayout(self.conn_tab.layout)
        self.run_tab.layout.removeWidget(self.run_device_table)
        self.run_tab.setLayout(self.run_tab.layout)
        
        ##############################
        #normal thread starting stuff#
        ##############################
        new_thread = Worker(function, *args)
        new_thread.signals.operations.connect(self.threadhandler)
        self.threadpool.start(new_thread)
        #####################################
        #end of normal thread starting stuff#
        #####################################
        
        self.conn_devices_table = QTableView()
        self.device_list = table_model(self.connected_devices)
        self.conn_devices_table.setModel(self.device_list)
        self.conn_tab.layout.addWidget(self.conn_devices_table,2,0,5,5)
        self.conn_tab.setLayout(self.conn_tab.layout)
        
        self.run_device_table = QTableView()
        self.device_list_run = table_model(self.connected_devices_run)
        self.run_device_table.setModel(self.device_list_run)
        self.run_tab.layout.addWidget(self.run_device_table,4,0,5,5)
        self.run_tab.setLayout(self.run_tab.layout)
        
    def testingthing(self):
        #for adding the table after the threads
        
        #This is dumb
        self.conn_devices_table = QTableView()
        self.connected_devices = pd.DataFrame([
            ["{server_ip}", "{server_port}", "Connections Tab","{current_mode}"],
            ],columns = ["Ipv4 Address", "Port", "Device State","Current Program"],
            index = ["Server"]
            )
        self.device_list = table_model(self.connected_devices)
        self.conn_devices_table.setModel(self.device_list)
        self.conn_tab.layout.addWidget(self.conn_devices_table,2,0,5,5)
        self.conn_tab.setLayout(self.conn_tab.layout)
        
        
        self.run_device_table = QTableView()
        self.connected_devices_run = pd.DataFrame([
            ["Running","{current_mode}","N/A"],
            ],columns = ["Device State","Current Program","Last Ping"],
            index = ["Server"]
        )
        self.device_list_run = table_model(self.connected_devices_run)
        self.run_device_table.setModel(self.device_list_run)
        self.run_tab.layout.addWidget(self.run_device_table,4,0,5,5)
        self.run_tab.setLayout(self.run_tab.layout)

        
    #note for any additions, is set to expect a tuple
    #sending an int closes the thread, unless the int is in a tuple (duh)
    def threadhandler(self, command_list):
        pass

def arbitrary_task(*args):
    print("Started Thread")
    while(True):
        pass

def main():
    global window
    app = QApplication([])
    window = Window()
    window.show()
    sys.exit(app.exec())
    
#call main to start program
main()

[SGaist]

Is your code based on that tutorial ?

[imissthecommandline]

@SGaist in part yes. Do you have a better tutorial or resource you reccomend?

[imissthecommandline]

sorry, i used this one: https://www.pythonguis.com/tutorials/multithreading-pyqt6-applications-qthreadpool/

[imissthecommandline]

ok, so what is happening is that every thread is making several calls to the table model every time the tab is opened. is there a way you recommend to prevent this? @SGaist

[SGaist]

You can implement a debouncer. For example, accumulate a certain amount of changes and only then signal that the model has changed to minimize the amount of time the views will query the model.
Also, don't forget to properly use lock mechanism to avoid writing to the same object from multiple different threads.