Why reading file in a separate thread is slowing down GUI?

CJha

Hi, I am reading a large file in a separate thread by using QThread::create method. My function to read the large file is in an anonymous namespace:

namespace
{
    void readFile()
    {
        QFile file{fileName}; // fileName is a constant QString
        if(file.open(QIODevice::ReadOnly)) {
            QTextStream stream{&file};
            const QString text = stream.readAll();
            textListMutex.lock();
            if(!text.isEmpty())
                textList = text.split('\n', Qt::SkipEmptyParts); // textList is a QStringList
            textListMutex.unlock();
        }
    }
}

Upon receiving the input to read the file, which is by clicking a QPushButton, I do the following:

auto thread = QThread::create(readFile);
connect(thread, &QThread::finished, thread, &QObject::deleteLater);
connect(thread, &QThread::destroyed, this, &Widget::readFileThreadDestroyed); // readFileThreadDestroyed() is a function to detect when reading is done
thread->start();

I check for QThread::currentThreadId() in both readFile() function and in my main class from where the command is issued. The function readFile() is being properly executed in a separate non-GUI thread. The GUI is not blocked but lagging severly i.e. it takes more than a few seconds (sometimes even 5 to 10 seconds) to respond to anything, but it does respond while the readFile() is still running in a separate thread, it's just that the response time is way slower than what it is when readFile() is not running in a separate thread. Why is this happening? Is there any way I could make my GUI thread respond faster while the readFile() is running in a separate thread? I tried changing the thread priority but that does not help at all.

CJha

@CJha My bad! solved it, it is because of QString QTextStream::readAll() function. The Qt document clearly states:

Avoid this function when working on large files, as it will consume a significant amount of memory

I assumed having enough memory and running this function in a separate thread would mean that it will work well without any problems but that is not the case. Instead, I am now using QString QTextStream::readLine(qint64 maxlen = 0) which makes the GUI smooth as butter ;)

JonB

@CJha
There may be some "general machine slowness" when doing this file reading, but I don't know of any reason why it should be "worse" if in a thread. I would start by (temporarily) commenting out the textListMutex code, need to eliminate possibility that it is waiting on a lock.

CJha

@JonB I commented out the textListMutex and still the result is the same. My CPU usage is around 10% (I am using a quad-core Intel i5 11th gen CPU) while the readFile() function is executing. The file I am reading is a .txt file with a size of ~39 MB.

CJha

@CJha My bad! solved it, it is because of QString QTextStream::readAll() function. The Qt document clearly states:

Avoid this function when working on large files, as it will consume a significant amount of memory

I assumed having enough memory and running this function in a separate thread would mean that it will work well without any problems but that is not the case. Instead, I am now using QString QTextStream::readLine(qint64 maxlen = 0) which makes the GUI smooth as butter ;)

SimonSchroeder

@CJha said in Why reading file in a separate thread is slowing down GUI?:

Instead, I am now using QString QTextStream::readLine(qint64 maxlen = 0) which makes the GUI smooth as butter ;)

Just a slight performance hint from my experience. Sure, your GUI is now running smoothly, but your separate thread could run a whole lot faster. Use QTextStream::readLineInto(QString*, qint64) instead. This can reuse an existing QString variable instead of allocating memory over and over again for each separate line.

piervalli

@SimonSchroeder said in Why reading file in a separate thread is slowing down GUI?:

Just a slight performance hint from my experience. Sure, your GUI is now running smoothly, but your separate thread could run a whole lot faster. Use QTextStream::readLineInto(QString*, qint64) instead.

Power of the community,
Thanks Thanks

CJha

@SimonSchroeder Thank you! That is really good advice, I will try it out now :)