GUI freezes even with multithreading
-
QT application GUI on Ubuntu 16.04 freezes.
What we are using -
Intel Celeron dual core 1.6GHz, RAM 2GB, 128gb SSD.We have used two smart cameras (ethernet).
We have used QNetworkmanager to grab images from the cameras. We have used QSocket for communicating with the Cameras (socket server). We are updating data (not images, only camera result data, very small) on MySQL server every 500ms. We have also used serial communication to communicate with a micro controller at 115200 baud rate. We use QSerialport for this. Frequency of serial data varies.The problem we used to face is that after prolonged use, sometimes, the GUI would freeze (screen would turn black and white, i.e application not responding). After waiting for some time, the GUI would recover and start running normally. This would happen randomly and not periodically. Sometimes the GUI would hang everyday, but sometimes, it would not hang for a week or two. We have checked that there are no memory leaks, no over heating, no noise. We then realised that the data coming in was a lot to process on the main thread. So to avoid this, we used Qthread. Now, we have separate threads for QNetworkmanager, Serial communication & Database writing. After this, the screen does not go black and white (not responding), but for some seconds, the screen stops functioning (no buttons can be pressed or no screen transition takes place). It looks like a long lag in the GUI operations. After multithreading, the application not responding problem has stopped but the screen still freezes for sometime. Now, sometimes, the application hangs even after starting it (at user login).
We thought that since each thread executes continuously and processes incoming data and passes it to the main thread at a very high frequency, the application becomes slow and hangs. So we added sleep delays of around 500ms to each thread. Unfortunately, even after doing this, the application froze for 45 seconds. The application started running properly after that. The application froze when a button was pressed that closes the login form and goes to the run form (at this time, all threads start). Here we think that the application took some time to start all threads. We would be grateful to receive help on this issue. thank you.
For your reference, I've attached a video link to show the freeze problem. Please watch the video till the end to see how the GUI recovers from the freeze.Constructor in main thread -
/**************************START Of Constructor*****************************************/ frm_run::frm_run(QWidget *parent) : QMainWindow(parent), ui(new Ui::frm_run) { ui->setupUi(this); try { this->setWindowFlags(Qt::FramelessWindowHint); // To remove screen Title bar ThreadMysql =new thread_mysql; //Thread for mysql data update ThreadMysql->start(); ThreadSerial = new thread_serial; //thread for serial recive data parce ThreadSerial->start(); ThreadBaumer = new thread_baumer; //Thread for Image grab from Web ThreadBaumer->start(); Login_name = dlg_login::current_user; Login_level = dlg_login::Current_level; qDebug()<<"level"<<Login_level; /********************Keyboard Connection with LineEdit**********/ lineEditkeyboard = new Keyboard; connect(ui->txt_light ,SIGNAL(selectionChanged()),this,SLOT(run_keyboard_lineEdit())); connect(ui->txt_shift_image ,SIGNAL(selectionChanged()),this,SLOT(run_keyboard_lineEdit())); connect(ui->txt_JobNumber ,SIGNAL(selectionChanged()),this,SLOT(run_keyboard_lineEdit())); connect(ui->txt_ProductID ,SIGNAL(selectionChanged()),this,SLOT(run_keyboard_lineEdit())); connect(ui->txt_Tollerence ,SIGNAL(selectionChanged()),this,SLOT(run_keyboard_lineEdit())); connect(ui->txt_Area_Threshold ,SIGNAL(selectionChanged()),this,SLOT(run_keyboard_lineEdit())); global::JobStatus = 0; QSqlQuery QRY; QString Query="select job_status,work_ticket_no,product_id from JOB_DETAILS WHERE job_status = '1' OR job_status = '2'"; if(QRY.exec(Query)) { while(QRY.next()) { global::JobStatus = QRY.value(QRY.record().indexOf("job_status")).toUInt(); global::WorktktNo = QRY.value(QRY.record().indexOf("work_ticket_no")).toString(); ProductID = QRY.value(QRY.record().indexOf("product_id")).toString(); } } Query="select * from MACHINE_SETTING WHERE id = 1 "; if(QRY.exec(Query)) { while(QRY.next()) { multiplier = (QRY.value(QRY.record().indexOf("multiplier")).toDouble()); EjectionStatus = (QRY.value(QRY.record().indexOf("ejection_status")).toUInt()); MaxConsBad = (QRY.value(QRY.record().indexOf("max_cons_bad")).toUInt()); Ejector_Distance = (QRY.value(QRY.record().indexOf("ejector_distance")).toUInt()); FeederToCam1Dist = (QRY.value(QRY.record().indexOf("feeder_to_cam1_dist")).toUInt()); FeederToCam2Dist = (QRY.value(QRY.record().indexOf("feeder_to_cam2_dist")).toUInt()); Exposure = (QRY.value(QRY.record().indexOf("exposure")).toUInt()); Cam2Exposure = (QRY.value(QRY.record().indexOf("cam2_exposure")).toUInt()); global::Cam1IP = QRY.value(QRY.record().indexOf("cam1_ip")).toString(); global::Cam2IP = QRY.value(QRY.record().indexOf("cam2_ip")).toString(); Pattern_Theshold = (QRY.value(QRY.record().indexOf("pattern_theshold")).toDouble()); Area_enable = (QRY.value(QRY.record().indexOf("area_enable")).toUInt()); flag_TopCamEnable = (QRY.value(QRY.record().indexOf("top_enable")).toInt()); flag_BtmCamEnable = (QRY.value(QRY.record().indexOf("bottom_enable")).toInt()); } } qDebug()<<"top_enable"<<flag_TopCamEnable<<flag_BtmCamEnable; TopCam_S = new Socket_Class; BtmCam_S = new Socket_Class; File_Class::send_log("Run Form Start"); connect(ThreadSerial,SIGNAL(rec_com_data()),this,SLOT(show_serial_data())); connect(ThreadBaumer,SIGNAL(geting_image_cam()),this,SLOT(GetImage())); connect(TopCam_S,&Socket_Class::client_status,this,&frm_run::get_top_status); connect(TopCam_S,&Socket_Class::client_data,this,&frm_run::get_top_data); connect(BtmCam_S,&Socket_Class::client_status,this,&frm_run::get_btm_status); connect(BtmCam_S,&Socket_Class::client_data,this,&frm_run::get_btm_data); connect(ThreadBaumer,SIGNAL(put_image_cam1()),this,SLOT(get_top_web())); connect(ThreadBaumer,SIGNAL(put_image_cam2()),this,SLOT(get_btm_web())); connect(ui->Image_Box,SIGNAL(Mouse_Pos()),this,SLOT(Mouse_current_pos())); connect(ui->Image_Box,SIGNAL(Mouse_Pressed()),this,SLOT(Mouse_Pressed())); connect(ui->Image_Box,SIGNAL(Mouse_Left()),this,SLOT(Mouse_Left())); if(flag_TopCamEnable == 1) { Cam1Sel = 1; } else { Cam1Sel = 0; } if(flag_BtmCamEnable == 1) { Cam2Sel = 1; } else { Cam2Sel = 0; } LoadDatabase(); // Load Parameters from Database if((global::JobStatus == 1)||(global::JobStatus == 2)) { send_data(); // Send parameter to microcontroller by Serial Port } ui->txt_shift_image->setReadOnly(true); ui->txt_light->setReadOnly(true); ui->txt_Tollerence->setReadOnly(true); if(Cam1Sel == 1) { TopCam_S->Cam_Connect(global::Cam1IP); // Connect to Socket Server } if(Cam2Sel == 1) { BtmCam_S->Cam_Connect(global::Cam2IP); // Connect to Socket Server } ui->Txt_MachineMode->setVisible(false); ui->txt_User->setEnabled(false); ui->txt_UserLevel->setEnabled(false); //ui->btn_Logout->setEnabled(false); // ui->btn_Finish->setEnabled(false); if(global::JobStatus != 0) { ui->txt_JobNumber->setEnabled(false); ui->chk_cam1->setEnabled(false); ui->chk_cam2->setEnabled(false); ui->btn_List->setVisible(false); if(global::JobStatus == 1) { #ifdef ADVANCE ui->btn_CodeType->setEnabled(true); #else ui->btn_CodeType->setEnabled(false); #endif } else { ui->btn_CodeType->setEnabled(false); } ui->btn_ROI->setEnabled(true); ui->btn_En_ROI->setEnabled(true); } else { ui->btn_CodeType->setEnabled(false); ui->btn_ROI->setEnabled(false); ui->btn_En_ROI->setEnabled(false); } ui->txt_ProductID->setEnabled(false); if((global::JobStatus == 1)||(global::JobStatus == 2)) { send_tcp_cam_Parameter(); // Send Socket Command to two Socket Server } qDebug()<<"End of funtion" <<Cam1Sel<<" "<<Cam2Sel; } catch(...) { //send_log("Mainwindow",1); } } /**************************END Of Constructor*****************************************/ --Our Thread run looks like this -
void function::run() { QMutex mutex; while(1) { mutex.lock(); if(this->Stop) { break; } // function processing mutex.unlock(); QThread::msleep(500); } }Would really appreciate help.
-
@dev_512
you're running 4 threads, 3 of them in a while true loop and asking yourself why the OS has trouble managing the tasks, do you at least have 4 cores - real ones not hyper threading stuff?I would suggest at the very least, put the processing threads into one and the Gui into the other, and read through this
https://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/
It will go a long way ;-)and for heavens sake do not use QThread::sleep, it will cause more problems than it will fix
-
Apart from what @J-Hilk said, first your MainWindow constructor is very heavy.
Your UI freezes means that GUI thread is busy. Why it is busy ? We need to find out.
Your MainWindows constructor is heavily loaded. It is following things.
- Starts thread
- Makes db query
- Connects with Camera
- Loading DB
All this could be taking around 45 seconds & i'm not surprised. Just to trouble shoot the issue, start Simple UI creation. Then start adding one-by-one functionality. This should give you the exact reason why UI freezes.
-
@J.Hil Thank you is much for your reply. I wanted to know how to avoid using sleep. Without using sleep, the threads will run very quickly compared to the main thread and emit a lot of data very rapidly. How to avoid sleep then? The GUI starts lagging and becomes unresponsive. Now that we are using sleep, we haven't noticed that problem yet.
-
@dev_512
that really depends on what you're actully doing inside the run function.
I would recommend reading through the block post I linked earlier, it's an old one but still valid. Since then the QThread-docu actually has a example of the worker approach.If you really have to stick with what you allready have I would suggest a QTimer.
- remove the while loop
- move the stuff inside the loop to its own function
- create a QTimer inside run, set the intvall to 500
- connect the timeout signal to the function
-
- The function will be executed inside the calling thread
- start the timer
- call exec() at the end of run
-
@J.Hilk Okay thank you, I am removing sleep delays, removing all threads and just making one GUI thread and another thread that will do all continuous functions such as serial, socket, QNetwork, etc. Thank you very much. Will let you know how it works out.
-
@J.Hilk hi, I have now formed only 1 thread apart from the main thread. In this thread, I have performed all the continuous functions and I have used QTimer instead of sleep now. I wanted to know if I have to assign a core of the processor to the thread. I don’t know if that needs to be done. Thank you.
-
You don't have to. It is not required. You can't directly unless you write some platform dependent code. Are you seeing any issue ?
-
@dheerendra no issues so far. Will update. Thank you.
-
@J.Hilk @dheerendra Hi, as you had suggested, we stopped using 4 threads and now are using just the main thread and another thread. We do all the continuous data handling functions in the other thread while GUI runs in the main thread. The system worked fine for 20-22 days (daily use). The system is used (in running state) almost 24*7 with lunch and tea breaks. After 20-22 days, the application became unresponsive (black and white screen in Ubuntu). Could the processor be the issue? It is a 1.6GHz dual core processor. Please help.
-
Just see whether process is still running ? Just check it is a issue of memory ? Are there any memory leaks in your program ?
-
@dheerendra No Sir, we have performed checks. no memory leaks. The free RAM memory stays constant even after an entire day. So we take and process data in the worker thread. After the processing, we pass the data to the main GUI thread. In case the worker thread hangs due to some reason, is it possible that the main thread will hang too? The worker thread does all the work while the main thread only receives and displays the result.
-
No main will not hang because of worker is hung. This can only happen if any dead lock arises. If you are using signal/slots without Qt::DirectConnection, it should not hang.
Now only way is to troubleshoot what is worker thread doing & why it is hung.
-
@dheerendra So just to check, we put an infinite while loop in the worker thread and started the system. The worker thread after processing data from serial communication and QImage function, generates signals and sends it to the main GUI thread. The Main thread only takes action after receiving these signals and displays the result on screen. Now when we added a while loop (infinite count) to the worker thread, the worker thread got busy performing that and stopped other tasks. Due to this, the worker thread stopped generating signals and hence, the GUI didn't display anything. However, the GUI did not freeze at all. The micro controller was sending data to the PC but the worker thread did not process anything since it was stuck in the while loop. We thought that this would freeze the main thread but the main thread (GUI) was completely responsive. Due to this, we think that something in the main thread hangs.
If the worker thread is stuck, the GUI thread does not become unresponsive. If we add the infinite counting while loop to the main thread, the GUI freezes immediately (not responding).
I am attaching the main thread so you can observe and suggest any changes. Please help. Thank you.#include "mainwindow.h" #include "ui_mainwindow.h" #include"serialcom.h" #include <QThread> #include <QDebug> #include "my_label.h" #include "baumer_class.h" #include "file_class.h" #include "mysql_class.h" #include "global.h" #include <QtSql> #include <QtSql/QSqlDatabase> #include <QtSql/QSqlQuery> #include "frm_run.h" MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), ui(new Ui::MainWindow) { ui->setupUi(this); this->setWindowFlags(Qt::FramelessWindowHint); //SerialCom::com_connect(); SerialCom::send_command(2,1); thread()->msleep(5); this->updateGeometry(); #ifdef ADVANCE ui->lbl_Version->setText(QString(global::VERSION + ".a")); #else // ui->lbl_Version->setText("v 1.2.7.b"); ui->lbl_Version->setText(QString(global::VERSION + ".b")); #endif; JobStatus = 0; QSqlQuery QRY; QString Query="select job_status,work_ticket_no,product_id from JOB_DETAILS WHERE job_status = '1' OR job_status = '2'"; if(QRY.exec(Query)) { while(QRY.next()) { JobStatus = QRY.value(QRY.record().indexOf("job_status")).toUInt(); WorktktNo = QRY.value(QRY.record().indexOf("work_ticket_no")).toString(); ProductID = QRY.value(QRY.record().indexOf("product_id")).toString(); } } Query="select * from MACHINE_SETTING WHERE id = 1 "; if(QRY.exec(Query)) { while(QRY.next()) { flag_TopCamEnable = (QRY.value(QRY.record().indexOf("top_enable")).toInt()); flag_BtmCamEnable = (QRY.value(QRY.record().indexOf("bottom_enable")).toInt()); } } update_timer = new QTimer; check_count = 0; connect(update_timer,SIGNAL(timeout()),this,SLOT(update_data())); if((flag_TopCamEnable == 1)&&(global::Top_SID_Connect ==false)) { QProcess process_1; process_1.startDetached("/bin/sh", QStringList()<< global::PATH_GET_SID_TOP); update_timer->start(15000); flag_top_connect = true; flag_btm_connect = false; ui->lbl_status->setText("Top Camera Connecting..."); } else if(global::Btm_SID_Connect == false) { QProcess process; process.startDetached("/bin/sh", QStringList()<< global::PATH_GET_SID_BTM); update_timer->start(15000); flag_top_connect = false; flag_btm_connect = true; ui->lbl_status->setText("Bottom Camera Connecting wait..."); } if(flag_TopCamEnable != 1 ) { global::Top_SID_Connect = true; } if(flag_BtmCamEnable != 1) { global::Btm_SID_Connect = true; } #ifdef PC ui->btn_login->setEnabled(true); #else if((global::Top_SID_Connect == true) &&(global::Btm_SID_Connect == true)) { ui->btn_login->setEnabled(true); } else { ui->btn_login->setText("Wait..."); ui->btn_login->setEnabled(false); } #endif } MainWindow::~MainWindow() { qDebug()<<"distructor"; delete ui; } void MainWindow::update_data() { try { bool flag_btm_skip = false; retry_count ++; if(flag_top_connect) { if(retry_count >= 2) { try { if(File_Class::read_file(global::PATH_SID_TOP_TEXT)) { if(File_Class::File_data != "") { global::Cam1_sid = File_Class::File_data.trimmed(); qDebug()<<"SID TOP"<<global::Cam1_sid; global::Top_SID_Connect = true; } else { global::Top_SID_Connect = false; } } else { global::Top_SID_Connect = false; } } catch(...) { global::Top_SID_Connect = false; } if(global::Btm_SID_Connect == false) { bottom_process_start(); retry_count = 0; flag_btm_skip = true; } else { update_timer->stop(); } } else { try { if(File_Class::read_file(global::PATH_SID_TOP_TEXT)) { if(File_Class::File_data != "") { global::Cam1_sid = File_Class::File_data.trimmed(); qDebug()<<"SID TOP"<<global::Cam1_sid; global::Top_SID_Connect = true; } else { global::Top_SID_Connect = false; } } else { global::Top_SID_Connect = false; } } catch(...) { global::Top_SID_Connect = false; } if(global::Top_SID_Connect == false) { QProcess process_1; process_1.startDetached("/bin/sh", QStringList()<< global::PATH_GET_SID_TOP); update_timer->start(15000); } else { if(global::Btm_SID_Connect == false) { bottom_process_start(); retry_count = 0; flag_btm_skip = true; } else { update_timer->stop(); } } } } else if(flag_btm_connect) { if(retry_count >= 2) { try { if(File_Class::read_file(global::PATH_SID_BTM_TEXT)) { if(File_Class::File_data != "") { global::Cam2_sid = File_Class::File_data.trimmed(); qDebug()<<"SID Btm"<<global::Cam2_sid; global::Btm_SID_Connect = true; } else { global::Btm_SID_Connect = false; } } else { global::Btm_SID_Connect = false; } } catch(...) { global::Btm_SID_Connect = false; } } else { try { if(File_Class::read_file(global::PATH_SID_BTM_TEXT)) { if(File_Class::File_data != "") { global::Cam2_sid = File_Class::File_data.trimmed(); qDebug()<<"SID Btm"<<global::Cam2_sid; global::Btm_SID_Connect = true; } else { global::Btm_SID_Connect = false; } } else { global::Btm_SID_Connect = false; } } catch(...) { global::Btm_SID_Connect = false; } if(global::Btm_SID_Connect == false) { QProcess process_2; process_2.startDetached("/bin/sh", QStringList()<< global::PATH_GET_SID_BTM); update_timer->start(15000); } } if((retry_count >= 2)||(global::Btm_SID_Connect == true)) { update_timer->stop(); } } QString str=""; if(flag_TopCamEnable != 0) { if(global::Top_SID_Connect == true) { str = "Top Camera Connected"; } else { if(retry_count >= 2) { str = "Top Camera unable to connect"; } else { str = "Top Camera Connecting"; } } } if(flag_BtmCamEnable != 0) { if(flag_btm_connect && (flag_btm_skip == false)) { if(global::Btm_SID_Connect == true) { str += " Bottom Camera Connected"; } else { if(retry_count == 2) { // update_timer->stop(); str += " Bottom Camera unable to connect"; } else { str += " Bottom Camera Connecting"; } } } } // str += " "+QString::number(retry_count); ui->lbl_status->setText(str); if(retry_count >= 2) { retry_count = 0; } flag_btm_skip = false; if((global::Top_SID_Connect == true)&&(global::Btm_SID_Connect == true)) { ui->btn_login->setEnabled(true); ui->btn_login->setText("Login"); } } catch(...) { } } void MainWindow::top_process_starrt() { } void MainWindow::bottom_process_start() { try { QProcess process; process.startDetached("/bin/sh", QStringList()<< global::PATH_GET_SID_BTM); check_count = 0; update_timer->start(5000); flag_top_connect = false; flag_btm_connect = true; } catch(...) { } } void MainWindow::on_btn_login_clicked() { DlgLogin = new dlg_login(this); DlgLogin->set_level("0",JobStatus); int dlg_int = DlgLogin->exec(); delete DlgLogin; if(dlg_int == QDialog::Accepted) { this->close(); FrmRun = new frm_run; FrmRun->show(); //delete this; // this->close(); } } void MainWindow::on_btn_Exit_clicked() { this->close(); } void MainWindow::on_btnFactorySetting_clicked() { try { DlgLogin = new dlg_login(this); DlgLogin->set_level("1",0); int dlg_int = DlgLogin->exec(); if(dlg_int == QDialog::Accepted) { QString str_level = ""; current_user = dlg_login::get_current_user(); QSqlQuery QRY; QString Query="select level from USER_DETAILS WHERE login_name = '"+current_user+"'"; if(Mysql_class::query.exec(Query)) { while(Mysql_class::query.next()) { str_level = Mysql_class::query.value(Mysql_class::query.record().indexOf("level")).toString(); } } if(str_level == "1") { frm_factory_setting::set_admin_flag(true); // flag_admin = true; } else { frm_factory_setting::set_admin_flag(false); // flag_admin = false; } this->close(); FrmFactory = new frm_factory_setting; FrmFactory->show(); } } catch(...) { } } void MainWindow::on_btn_Report_clicked() { DlgReport = new dlg_report(this); DlgReport->exec(); } void MainWindow::on_btn_shut_down_clicked() { try { QProcess process_1; process_1.startDetached("/bin/sh", QStringList()<< global::PATH_SHUTDOWN); } catch(...) { } } void MainWindow::on_btn_retry_connection_clicked() { try { retry_count = 0; if((global::Top_SID_Connect == false )||(global::Btm_SID_Connect == false)) { if(global::Top_SID_Connect == false) { QProcess process; process.startDetached("/bin/sh", QStringList()<< global::PATH_GET_SID_TOP); check_count = 0; retry_count = 0; update_timer->start(5000); flag_top_connect = true; flag_btm_connect = false; ui->lbl_status->setText(" Top Camera Connecting wait..."); } else if(global::Btm_SID_Connect == false) { // on_btn_Cam2_clicked(); bottom_process_start(); ui->lbl_status->setText(" Bottom Camera Connecting wait..."); } // else // { // ui->lbl_status->setText("Top Camera Connected and Bottom Camera Connected"); // } } } catch(...) { } }The application froze without any input from the user (the system was performing normal tasks of getting images from the camera every 750ms and updating data from serial count).
We have also kept a Timer in the main thread which tells the worker thread to get an image. -
Why are doing the infinite while in threads. If you do this definitely, main thread or any other thread will be unresponsive.
-
@dheerendra Yes we were doing that just to check if main thread hangs because of worker thread getting stuck. (we used to while thread to forcefully hang the worker thread). Did you find any problem in the main thread code?
-
@dheerendra At this point, we are fairly certain that the problem is in the main thread code. I have uploaded the main thread code in the previous comment. Please suggest any necessary changes that could solve the issue. thank you.
