Qt6 Windows slower than Qt6 Ubuntu

JonB

@Joe-von-Habsburg
Is the difference due to the QNetworkAccessManager _manager; or to the QNetworkReply *reply = _manager.get(_request);?
In the bad case you might connect to reply->destroyed() to make sure it is indeed getting freed.

Joe von Habsburg

@JonB

According to my tests, if I rebuild the NetworkManager, my memory usage does not increase. But this time my receiving time is slowing down.

If I do not recreate the NetworkManager, my reception time decreases but memory usage increases.

connect(reply, &QNetworkReply::destroyed, this, &DataReceiver::replyDeleted);

it's work, reply deleted

JonB

@Joe-von-Habsburg
In that case please replace the second one's QNetworkReply *reply = _manager.get(_request); with the first one's reply = _manager.get(_request); (I assume reply is a member variable, this->reply) and confirm. Then the only difference is the local QNetworkAccessManager _manager versus a class member one.

Joe von Habsburg

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

connect(reply, &QNetworkReply::destroyed, this, &DataReceiver::replyDeleted);

sorry for that. this is work only when I stop my while loop. How can force it ?

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

 void DataReceiver::start()
 {
     _connection++;
     if(_connection > 1)
         return;
 
     qDebug() << "Starting take data";
     _takeData = true;
     while(_takeData){
         QDateTime time1 = QDateTime::currentDateTime();
         _sample = getData(54664);
         QDateTime time3 = QDateTime::currentDateTime();
         if(_takeIQData)
             _iq = getData(54665);
         QDateTime time2 = QDateTime::currentDateTime();
         qDebug() << "sample : " << time3 - time1 << " iq : " << time2 - time3 << "time :  << time2 - time1;
        emit sendData(_sample, _iq);
     }
     qDebug() << "Finished taking data";
 }

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

n that case please replace the second one's QNetworkReply *reply = _manager.get(_request); with the first one's reply = _manager.get(_request); (I assume reply is a member variable, this->reply) and confirm. Then the only difference is the local QNetworkAccessManager _manager versus a class member one.

I change it.
_reply and _manager class member now

QByteArray DataReceiver::getData(int port)
{
    QString url = QString("http://localhost:%1/sample").arg(port);
    QUrl _apiUrl(url);
    QNetworkRequest _request(_apiUrl);
    QEventLoop _loop;
    _reply = _manager.get(_request);
    connect(_reply, &QNetworkReply::finished, &_loop, &QEventLoop::quit);
    connect(_reply, &QNetworkReply::destroyed, this, &DataReceiver::replyDeleted);
    _loop.exec();
    if (_reply->error() == QNetworkReply::NoError) {
        QByteArray responseData = _reply->readAll();
        _reply->deleteLater();
        return  responseData;
    } else {
        QString _error = tr("Error");
        QString _spectrum_error = tr("Spectrum error.");
        QMessageBox::critical(nullptr, _error, _spectrum_error);
        stop();
    }
    _reply->deleteLater();
    return {};
}

The problem still exists.

I think lots of reply are created and they only delete I stop my loop.

JonB

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

I think lots of reply are created and they only delete I stop my loop.

I do not follow this. One reply per _manager.get(_request), and you have checked that is getting destroyed.

Supposedly, the only difference now is that in good case you create and destroy a local QNetworkAccessManager _manager each time (hence slower) while in bad case you reuse member variable. Right?

In that case can only suggest you try playing with stuff to see if it makes any difference in the bad case. QNAM has a couple of caches you can clear (see docs), try those. Try calling its setAutoDeleteReplies(true). Try making the member variable a QNetworkAccessManager *_manager which you new and deleteLater() or delete and instead create a new one in the member variable each time its destroyed for use next time. That would "emulate" having it as a local variable which gets automatically created & destroyed in the good case.

Other than that I am out of ideas. maybe someone expert will comment.

Joe von Habsburg

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

One reply per _manager.get(_request), and you have checked that is getting destroyed.

I can't delete _reply after get data.

_reply->deleteLater();

I used connect function, I saw this result: when I stop loop, _replys will be deleted (lots of delete).

connect(_reply, &QNetworkReply::destroyed, this, &DataReceiver::replyDeleted);
void DataReceiver::replyDeleted()
{
    qDebug() << "reply deleted.";
}

Result :

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

Supposedly, the only difference now is that in good case you create and destroy a local QNetworkAccessManager _manager each time (hence slower) while in bad case you reuse member variable. Right?

Yes you right, if I do not use as local member (_manager), response time slow, but I use as local member, memory leak.

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

ry calling its setAutoDeleteReplies(true). Try making the member variable a QNetworkAccessManager *_manager which you new and deleteLater()

QByteArray DataReceiver::getData(int port)
{
    QString url = QString("http://localhost:%1/sample").arg(port);
    QUrl _apiUrl(url);
    QNetworkRequest _request(_apiUrl);
    QEventLoop _loop;
    _manager = new QNetworkAccessManager();
    _manager->setAutoDeleteReplies(true);
    _reply = _manager->get(_request);
    connect(_reply, &QNetworkReply::finished, &_loop, &QEventLoop::quit);
    connect(_reply, &QNetworkReply::destroyed, this, &DataReceiver::replyDeleted);
    _loop.exec();
    if (_reply->error() == QNetworkReply::NoError) {
        QByteArray responseData = _reply->readAll();
        _reply->deleteLater();
        _manager->deleteLater();
        return  responseData;
    } else {
        QString _error = tr("Error");
        QString _spectrum_error = tr("Spectrum error.");
        QMessageBox::critical(nullptr, _error, _spectrum_error);
        stop();
    }
    _reply->deleteLater();
    return {};
}

I added them my function, but now slow time and memory leak exist together.

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

Other than that I am out of ideas. maybe someone expert will comment.

Thank you, Anyone can help me, I will be honered.

Joe von Habsburg

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

setAutoDeleteReplies(true)

this is crashed my program when I do something.

if I do it :

_manager.setAutoDeleteReplies(true);

memory and speed ok, but program crashed anytime , when I do anything. because reply deleted. why ?

JonB

@Joe-von-Habsburg
I would guess you need to not delete it yourself now (_reply->deleteLater()) since it is being auto-deleted on QNetworkReply::finished?

However if it is being auto-deleted on finished then, because you do _loop.exec(); which waits for finished I would guess you can no longer do _reply->readAll() after the loop, that sounds like crash. You could instead do your data reading inside the loop instead of afterwards.

I really don't know, and have never used any of this stuff :) It was suggestions for narrowing down where the "leakage" might be. But you need someone else if you need someone who truly knows about it.

Joe von Habsburg

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

You could instead do your data reading inside the loop instead of afterwards.

can you give me code example ?

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

But you need someone else if you need someone who truly knows about it.

Yes but who will be :D

JonB

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

can you give me code example ?

loop.exec() is just a convenience, which blocks till finished(), making your call "synchronous". Instead you can deal with all the data which arrives when it arrives (instead of after finished()) by attaching a slot to _reply->readyRead() signal.

Of course, if you remove _loop.exec() from getData() it will exit while network stuff is still going on, you must not have local variables which go out of scope. You will have to do quite a bit of code reorganizing. And it might be to no avail, I don't know what it will reveal or whether it will solve anything.

Joe von Habsburg

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

loop.exec() is just a convenience, which blocks till finished(), making your call "synchronous". Instead you can deal with all the data which arrives when it arrives (instead of after finished()) by attaching a slot to _reply->readyRead() signal.

I changed my code.

void DataReceiver::start()
{
    if(!_takeData)
        return;

    QDateTime time1 = QDateTime::currentDateTime();
    getDataSample(); //_sample = getData(54664);
    QDateTime time3 = QDateTime::currentDateTime();
    if(_takeIQData)
        getDataIQ(); //_iq = getData(54665);
    QDateTime time2 = QDateTime::currentDateTime();
    qDebug() << "sample : " << time3 - time1 << " iq : " << time2 - time3 << "time : " << time2 - time1;
}

void DataReceiver::getDataSample()
{
    QString url = QString("http://localhost:%1/sample").arg(54664);
    QUrl _apiUrl(url);
    QNetworkRequest _request(_apiUrl);
    _replySample = _managerSample.get(_request);
    connect(_replySample, &QNetworkReply::readyRead, this, &DataReceiver::readSample);
}

void DataReceiver::getDataIQ()
{
    QString url = QString("http://localhost:%1/sample").arg(54665);
    QUrl _apiUrl(url);
    QNetworkRequest _request(_apiUrl);
    _replyIQ = _managerIQ.get(_request);
    connect(_replyIQ, &QNetworkReply::readyRead, this, &DataReceiver::readIQ);
}

void DataReceiver::readSample()
{
    _sample = _replySample->readAll();
    _replySample->deleteLater();
    if(!_takeIQData){
        emit sendData(_sample, _iq);
        _sample.clear();
        _iq.clear();
        start();
    }

}

void DataReceiver::readIQ()
{
    _iq = _replyIQ->readAll();
    _replyIQ->deleteLater();
    if(_takeIQData){
        emit sendData(_sample, _iq);
        _sample.clear();
        _iq.clear();
        start();
    }
}

After a while crashing :(

Joe von Habsburg

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

After a while crashing :(

I found the reason. Because readReady() sometimes receives incomplete data.

jsulm

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

sometimes receives incomplete data

Yes, because there is no guarantee that you get all data in one piece. That's why you need to buffer incoming data until you received a whole package of data.

Christian Ehrlicher

@jsulm... or use the requestFinished signal 🙂

Joe von Habsburg

@jsulm said in Qt6 Windows slower than Qt6 Ubuntu:

Yes, because there is no guarantee that you get all data in one piece. That's why you need to buffer incoming data until you received a whole package of data.

Yes, I try downloadProgress signal like that:

void DataReceiver::downloadProgressSample(qint64 bytesReceived, qint64 bytesTotal)
{
    if(bytesReceived == bytesTotal){
        _isSampleOk = true;
        _sampleLen = bytesReceived;
    }
    else{
        _isSampleOk = false;
        _sampleLen = 0;
    }
}

but so many times, my QByteArray's length and bytesReceived are not equal.... and my program slowing for that.

@Christian-Ehrlicher said in Qt6 Windows slower than Qt6 Ubuntu:

or use the requestFinished signal

I could not see requestFinished signal on documentation. Do you mean say finished signal ?

JonB

@Joe-von-Habsburg
Exactly as my colleagues have replied above for readyRead().

If you are still pursuing the difference in memory usage you report. A couple of points about your code:

You have a couple of calls to getData() inside some loop, while (_takeData). Your getData() uses (or used to use, you may have changed that now) _reply->deleteLater(). Although you have a QEventLoop::exec() call, which will allow signals/slots to flow, this is not the top-level Qt event loop. I believe deleteLater() causes deferred deletes, meaning that the memory is not actually released until the top-level event loop is re-entered. You should not use a while loop like this for your tests: allow the top-level Qt event loop to be re-entered.

getData() returns a QByteArray. We do not know what your code does with that result: for all we know you retain that somewhere, and that would eat up memory.

To investigate properly you should produce a minimal reproducible example with "good" code, to be sure what you are looking at.

jsulm

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

Yes, I try downloadProgress signal like that

What does this have to do with buffering incoming data?
To get a simpler solution follow @Christian-Ehrlicher suggestion.

JonB

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

but so many times, my QByteArray's length and bytesReceived are not equal.... and my program slowing for that.

I don't understand your point or question here? If you readyRead() as you go along or use QNetworkReply::downloadProgress() you will (almost certainly) see data arrive in "chunks", i.e. multiple calls until all the data is received. That is how the data is being transmitted, and is quite normal/expected.

As I wrote earlier, if you do not want to handle partial data as it arrives, and are using reply auto-delete so you cannot access it after it has finished, you can place a slot on QNetworkReply::finished and readAll() the data there in one go, before allowing the reply to be deleted.

Joe von Habsburg

@jsulm said in Qt6 Windows slower than Qt6 Ubuntu:

What does this have to do with buffering incoming data?
To get a simpler solution follow @Christian-Ehrlicher suggestion.

I could not see requestFinished signal on documentation. Do you mean say "finished" signal ?

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

don't understand your point or question here?

My question is that. if I use "finished" signal memory leak.

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

_reply->readyRead()

You suggested to me "readyRead" signal but now, I could not readyAll because, incomplated data is received.

For example, data's size must be 350000, but i receive 348000 or less. As a result of for that I crashed.

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

QNetworkReply::downloadProgress()

I try check will have arrived data size use with "downloadProgress" signal and my QByteArray data size is equal ? but so many times thay are not.

How can I wait all data complate without use "finished" signal because memory leak?

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

As I wrote earlier, if you do not want to handle partial data as it arrives, and are using reply auto-delete so you cannot access it after it has finished, you can place a slot on QNetworkReply::finished and readAll() the data there in one go, before allowing the reply to be deleted.

I don't understand. Can you give me code example ?

jsulm

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

You suggested to me "readyRead" signal but now, I could not readyAll because, incomplated data is received

As already suggested: you need to accumulate incoming data in a buffer. readyRead() signal can be emited several times until you get everything.

"Do you mean say "finished" signal ?" - yes
https://doc.qt.io/qt-6/qnetworkreply.html#finished
It even explains: "In particular, if no calls to read() were made as a result of readyRead(), a call to readAll() will retrieve the full contents in a QByteArray."