Qt6 Windows slower than Qt6 Ubuntu

JonB

@Joe-von-Habsburg
I would guess you need to not delete it yourself now (_reply->deleteLater()) since it is being auto-deleted on QNetworkReply::finished?

However if it is being auto-deleted on finished then, because you do _loop.exec(); which waits for finished I would guess you can no longer do _reply->readAll() after the loop, that sounds like crash. You could instead do your data reading inside the loop instead of afterwards.

I really don't know, and have never used any of this stuff :) It was suggestions for narrowing down where the "leakage" might be. But you need someone else if you need someone who truly knows about it.

Joe von Habsburg

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

You could instead do your data reading inside the loop instead of afterwards.

can you give me code example ?

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

But you need someone else if you need someone who truly knows about it.

Yes but who will be :D

JonB

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

can you give me code example ?

loop.exec() is just a convenience, which blocks till finished(), making your call "synchronous". Instead you can deal with all the data which arrives when it arrives (instead of after finished()) by attaching a slot to _reply->readyRead() signal.

Of course, if you remove _loop.exec() from getData() it will exit while network stuff is still going on, you must not have local variables which go out of scope. You will have to do quite a bit of code reorganizing. And it might be to no avail, I don't know what it will reveal or whether it will solve anything.

Joe von Habsburg

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

loop.exec() is just a convenience, which blocks till finished(), making your call "synchronous". Instead you can deal with all the data which arrives when it arrives (instead of after finished()) by attaching a slot to _reply->readyRead() signal.

I changed my code.

void DataReceiver::start()
{
    if(!_takeData)
        return;

    QDateTime time1 = QDateTime::currentDateTime();
    getDataSample(); //_sample = getData(54664);
    QDateTime time3 = QDateTime::currentDateTime();
    if(_takeIQData)
        getDataIQ(); //_iq = getData(54665);
    QDateTime time2 = QDateTime::currentDateTime();
    qDebug() << "sample : " << time3 - time1 << " iq : " << time2 - time3 << "time : " << time2 - time1;
}

void DataReceiver::getDataSample()
{
    QString url = QString("http://localhost:%1/sample").arg(54664);
    QUrl _apiUrl(url);
    QNetworkRequest _request(_apiUrl);
    _replySample = _managerSample.get(_request);
    connect(_replySample, &QNetworkReply::readyRead, this, &DataReceiver::readSample);
}

void DataReceiver::getDataIQ()
{
    QString url = QString("http://localhost:%1/sample").arg(54665);
    QUrl _apiUrl(url);
    QNetworkRequest _request(_apiUrl);
    _replyIQ = _managerIQ.get(_request);
    connect(_replyIQ, &QNetworkReply::readyRead, this, &DataReceiver::readIQ);
}

void DataReceiver::readSample()
{
    _sample = _replySample->readAll();
    _replySample->deleteLater();
    if(!_takeIQData){
        emit sendData(_sample, _iq);
        _sample.clear();
        _iq.clear();
        start();
    }

}

void DataReceiver::readIQ()
{
    _iq = _replyIQ->readAll();
    _replyIQ->deleteLater();
    if(_takeIQData){
        emit sendData(_sample, _iq);
        _sample.clear();
        _iq.clear();
        start();
    }
}

After a while crashing :(

Joe von Habsburg

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

After a while crashing :(

I found the reason. Because readReady() sometimes receives incomplete data.

jsulm

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

sometimes receives incomplete data

Yes, because there is no guarantee that you get all data in one piece. That's why you need to buffer incoming data until you received a whole package of data.

Christian Ehrlicher

@jsulm... or use the requestFinished signal 🙂

Joe von Habsburg

@jsulm said in Qt6 Windows slower than Qt6 Ubuntu:

Yes, because there is no guarantee that you get all data in one piece. That's why you need to buffer incoming data until you received a whole package of data.

Yes, I try downloadProgress signal like that:

void DataReceiver::downloadProgressSample(qint64 bytesReceived, qint64 bytesTotal)
{
    if(bytesReceived == bytesTotal){
        _isSampleOk = true;
        _sampleLen = bytesReceived;
    }
    else{
        _isSampleOk = false;
        _sampleLen = 0;
    }
}

but so many times, my QByteArray's length and bytesReceived are not equal.... and my program slowing for that.

@Christian-Ehrlicher said in Qt6 Windows slower than Qt6 Ubuntu:

or use the requestFinished signal

I could not see requestFinished signal on documentation. Do you mean say finished signal ?

JonB

@Joe-von-Habsburg
Exactly as my colleagues have replied above for readyRead().

If you are still pursuing the difference in memory usage you report. A couple of points about your code:

You have a couple of calls to getData() inside some loop, while (_takeData). Your getData() uses (or used to use, you may have changed that now) _reply->deleteLater(). Although you have a QEventLoop::exec() call, which will allow signals/slots to flow, this is not the top-level Qt event loop. I believe deleteLater() causes deferred deletes, meaning that the memory is not actually released until the top-level event loop is re-entered. You should not use a while loop like this for your tests: allow the top-level Qt event loop to be re-entered.

getData() returns a QByteArray. We do not know what your code does with that result: for all we know you retain that somewhere, and that would eat up memory.

To investigate properly you should produce a minimal reproducible example with "good" code, to be sure what you are looking at.

jsulm

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

Yes, I try downloadProgress signal like that

What does this have to do with buffering incoming data?
To get a simpler solution follow @Christian-Ehrlicher suggestion.

JonB

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

but so many times, my QByteArray's length and bytesReceived are not equal.... and my program slowing for that.

I don't understand your point or question here? If you readyRead() as you go along or use QNetworkReply::downloadProgress() you will (almost certainly) see data arrive in "chunks", i.e. multiple calls until all the data is received. That is how the data is being transmitted, and is quite normal/expected.

As I wrote earlier, if you do not want to handle partial data as it arrives, and are using reply auto-delete so you cannot access it after it has finished, you can place a slot on QNetworkReply::finished and readAll() the data there in one go, before allowing the reply to be deleted.

Joe von Habsburg

@jsulm said in Qt6 Windows slower than Qt6 Ubuntu:

What does this have to do with buffering incoming data?
To get a simpler solution follow @Christian-Ehrlicher suggestion.

I could not see requestFinished signal on documentation. Do you mean say "finished" signal ?

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

don't understand your point or question here?

My question is that. if I use "finished" signal memory leak.

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

_reply->readyRead()

You suggested to me "readyRead" signal but now, I could not readyAll because, incomplated data is received.

For example, data's size must be 350000, but i receive 348000 or less. As a result of for that I crashed.

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

QNetworkReply::downloadProgress()

I try check will have arrived data size use with "downloadProgress" signal and my QByteArray data size is equal ? but so many times thay are not.

How can I wait all data complate without use "finished" signal because memory leak?

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

As I wrote earlier, if you do not want to handle partial data as it arrives, and are using reply auto-delete so you cannot access it after it has finished, you can place a slot on QNetworkReply::finished and readAll() the data there in one go, before allowing the reply to be deleted.

I don't understand. Can you give me code example ?

jsulm

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

You suggested to me "readyRead" signal but now, I could not readyAll because, incomplated data is received

As already suggested: you need to accumulate incoming data in a buffer. readyRead() signal can be emited several times until you get everything.

"Do you mean say "finished" signal ?" - yes
https://doc.qt.io/qt-6/qnetworkreply.html#finished
It even explains: "In particular, if no calls to read() were made as a result of readyRead(), a call to readAll() will retrieve the full contents in a QByteArray."

Joe von Habsburg

@jsulm said in Qt6 Windows slower than Qt6 Ubuntu:

As already suggested: you need to accumulate incoming data in a buffer. readyRead() signal can be emited several times until you get everything.

Hmm now I should not assign reply->readAll(), I should append :). I will try it.

_sample.append(_replySample->readAll());

@jsulm said in Qt6 Windows slower than Qt6 Ubuntu:

"Do you mean say "finished" signal ?" - yes

I cannot use it because memory leak.

JonB

@Joe-von-Habsburg
As @jsulm has said. With QNetworkAccessManager::setAutoDeleteReplies(true) set you should be able to use either of the following approaches:

QByteArray _bytesRead;    // class member variable

_bytesRead.clear();
connect(_reply, &QNetworkReply::finished, this, &Class::onFinished);
_reply = _manager.get(_request);

void Class::onFinished()
{
    _bytesRead = _reply->readAll();    // read all bytes in one go at the end, just before `_reply` gets auto-deleted
}

or

QByteArray _bytesRead;    // class member variable

_bytesRead.clear();
connect(_reply, &QNetworkReply::readyRead, this, &Class::onReadyRead);
connect(_reply, &QNetworkReply::finished, this, &Class::onFinished);
_reply = _manager.get(_request);

void Class::onReadyRead()
{
    _bytesRead += _reply->readAll();    // *append* this time's bytes read to buffer
}

void Class::onFinished()
{
    // I think `_bytesRead` should contain all data by now, when reply has finished
    // If not call `_bytesRead += _reply->readAll();` or `onReadyRead()` one last time
}

[Edit: Fixed code highlighting ~kshegunov]

jsulm

@Joe-von-Habsburg said in Qt6 Windows slower than Qt6 Ubuntu:

cannot use it because memory leak.

There should not be memory leak unless your code is wrong.
Simply connect a slot to finished() signal and in that slot call readAll() and deleteLater() on the reply.

Joe von Habsburg

@JonB said in Qt6 Windows slower than Qt6 Ubuntu:

QByteArray _bytesRead; // class member variable

_bytesRead.clear();
connect(_reply, &QNetworkReply::readyRead, this, &Class::onReadyRead);
connect(_reply, &QNetworkReply::finished, this, &Class::onFinished);
_reply = _manager.get(_request);

void Class::onReadyRead()
{
_bytesRead += _reply->readAll(); // append this time's bytes read to buffer
}

void Class::onFinished()
{
// I think _bytesRead should contain all data by now, when reply has finished
// If not call _bytesRead += _reply->readAll(); or onReadyRead() one last time
}

its work !!! Thank you so much

JonB

@Joe-von-Habsburg
Do your timings/memory consumption with this asynchronous approach (i.e. no _loop.exec()) and not with that original while() loop. (If you want to run it more than once, after you get finished() on one start the next one there or on a QTimer::singleShot().) See whether you still get bad performance on one versus the other.

Joe von Habsburg

@JonB

Last edit my code :

DataReceiver::DataReceiver(QObject *parent)
   : QObject{parent}
{
   _manager.setAutoDeleteReplies(true);
}
void DataReceiver::start()
{
   _connection++;
   if(_connection > 1)
       return;

   _takeData = true;
   run();
}

void DataReceiver::stop()
{
   _takeData = false;
   _connection = 0;
   _data.clear();
}


void DataReceiver::getData()
{
   _data.clear();
   QString url = QString("http://localhost:%1/sample").arg(_port);
   QUrl _apiUrl(url);
   QNetworkRequest _request(_apiUrl);
   _reply = _manager.get(_request);
   connect(_reply, &QNetworkReply::readyRead, this, &DataReceiver::onReadReady);
   connect(_reply, &QNetworkReply::finished, this, &DataReceiver::onFinished);
}


void DataReceiver::run()
{
   if(!_takeData)
       return;

   getData();
}

void DataReceiver::onReadReady()
{
   _data += _reply->readAll();
}

void DataReceiver::onFinished()
{
   emit sendData(_data);
   run();
}

its working. time slow down from 65ms to 165ms but its working.