Never ending local Event Loop

Ekia

Hi everyone.

I'm posting because I'm facing an issue on some Mac machines (even though I'm not sure that it's OS dependent) where I'm apparently stuck in a local event loop that I created (Qt 5.15). Of course, I could not reproduce the issue on my own Mac, else it would have been too easy :)
The context is that I have a class that contains a QNetworkAccessManager and that provides a way to run a single POST request in a synchronous way using a local QEventLoop to make sure that the UI of the software is not frozen while we are waiting for the request to finish (I think that it's a bad idea, but the soft heavily relies on that, so that's not something I could change without a major redesign).

In the vast majority of cases, it works well, but sometimes, it seems to be "randomly" freezing on _localEvtLoop.exec(), as if the timeout that I've tried to set to return from the local loop exec() call was not working.

Here is a lighter version of my class implementation, if you could check it and give me some recommendations or explanations of what I'm doing wrong, that would be very much appreciated :)

Thanks in advance!

NetRequest::NetRequest() : QObject()
{
    _networkAccessManager = new QNetworkAccessManager(this);

    connect(_networkAccessManager, &QNetworkAccessManager::proxyAuthenticationRequired, this, &NetRequest::proxyAuthenticationRequired);
    connect(_networkAccessManager, &QNetworkAccessManager::sslErrors, this, &NetRequest::sslErrors);
}

NetRequest::~NetRequest()
{
    if (_timer)
    {
        _timer->stop();
        disconnect(_timer);
        delete _timer;
    }
    disconnect(_reply);
    disconnect(_networkAccessManager);

    delete _networkAccessManager;
}

bool NetRequest::post(const QString& urlStr, const QByteArray& data)
{
    QUrl url(urlStr);
    QNetworkRequest request(url);
    request.setAttribute(QNetworkRequest::RedirectPolicyAttribute, QNetworkRequest::NoLessSafeRedirectPolicy);
    //request.setTransferTimeout(10000);
    
    _reply = _networkAccessManager->post(request, data);
    bool retFlag = false;
    if (_reply)
    {
        connect(_reply, &QNetworkReply::errorOccurred, this, &NetRequest::error);
        connect(_reply, &QNetworkReply::finished, this, &NetRequest::requestFinished);
        connect(_reply, &QNetworkReply::sslErrors, this, &NetRequest::requestSslErrors);

        retFlag = readWithTimeout();
    }

    return retFlag;
}

bool NetRequest::readWithTimeout()
{
    bool retFlag = false;

    _timer = new QTimer(this);
    connect(_timer, &QTimer::timeout, this, &NetRequest::onTimerTimeout);
    _timer->start(10000); // 10 sec
    _timer->setSingleShot(true);

    int loopExitCode = _localEvtLoop.exec(QEventLoop::ExcludeUserInputEvents); // Sometimes, it's stuck here
    if (loopExitCode == 0)
    {
        _dataRead = _reply->readAll();
        retFlag = true;
    }
    return retFlag;
}

void NetRequest::onTimerTimeout()
{
    // In some cases, for some reason, I don't enter this method even after way more than the 10 sec timeout
    _localEvtLoop.exit(-1);
}

void NetRequest::sslErrors(QNetworkReply* reply, const QList<QSslError>& errors)
{
    for (int i = 0; i < errors.size(); ++i) 
    {
        QSslError::SslError errType = errors.at(i).error();
        if (errType == QSslError::SelfSignedCertificate || errType == QSslError::SelfSignedCertificateInChain)
        {
            reply->ignoreSslErrors();
            return;
        }
    }
}

// QNetworkReply signals
void NetRequest::requestFinished()
{
    if (_localEvtLoop.isRunning())
        _localEvtLoop.exit(0);
}

void NetRequest::error(QNetworkReply::NetworkError code)
{
    if (_localEvtLoop.isRunning())
        _localEvtLoop.exit(-1);
}

void NetRequest::requestSslErrors(const QList<QSslError>& errors)
{
    for (int i = 0; i < errors.size(); ++i) 
    {
        QSslError::SslError errType = errors.at(i).error();
        if (errType == QSslError::SelfSignedCertificate || errType == QSslError::SelfSignedCertificateInChain)
        {
            reply->ignoreSslErrors();
            return;
        }
    }
}

void NetRequest::proxyAuthenticationRequired(const QNetworkProxy& proxy, QAuthenticator* authenticator)
{
    // (...)
}

Axel Spoerl

Put qDebug() << Q_FUNC_INFO << code; in the error() method, and the same (without << code) into requestFinished().
If the debug statements are hit before the freeze, something is blocking the event loop.

SGaist

Hi,

If memory serves well, another possible scenario is that one of the events is handled by the main event loop rather than this one.

Can you tell us the implication in your application to change this request to asynchronous ? (just curiosity)

SimonSchroeder

You might have a problem with the network reply finishing before the local event loop is started. You need to start the event loop before you post the request. (It does not make fully sense because nobody could handle requestFinished yet. Is the slot posted to the main event loop before the the local event loop started? Would the local event loop not handle the queued slots from the main event loop?)

I'm very close to suggesting to use a solution with processEvents()...

BTW, if you exclude user input events the UI will not be very responsive (only reason would be a progress bar/dialog).

Ekia

@Axel-Spoerl said in Never ending local Event Loop:

Put qDebug() << Q_FUNC_INFO << code; in the error() method, and the same (without << code) into requestFinished().
If the debug statements are hit before the freeze, something is blocking the event loop.

I could not reproduce it on my side, but from the application logs, I'm sure that none of these 2 methods has been called before the freeze.

@SGaist said in Never ending local Event Loop:

Hi,

If memory serves well, another possible scenario is that one of the events is handled by the main event loop rather than this one.

Could you tell me more about that? I vaguely remember that I saw a comment around that while searching on internet, but I have to admit that I didn't fully understand it. Actually, I'm realizing that I don't know how Qt is deciding which of the 2 event loops (the main one and the local one) will be used.

@SGaist said in Never ending local Event Loop:

Can you tell us the implication in your application to change this request to asynchronous ? (just curiosity)

Maybe I'm over-estimating the cost, but actually I have many calls that were originally corresponding to the application code being called synchronously through functions. Now, many of these calls have been replaced to calls to micro-services, and it seemed easier at that time to simply mimic the synchronous behavior of functions by making these requests synchronous too.

@SimonSchroeder said in Never ending local Event Loop:

You might have a problem with the network reply finishing before the local event loop is started. You need to start the event loop before you post the request. (It does not make fully sense because nobody could handle requestFinished yet. Is the slot posted to the main event loop before the the local event loop started? Would the local event loop not handle the queued slots from the main event loop?)

I'm very close to suggesting to use a solution with processEvents()...

BTW, if you exclude user input events the UI will not be very responsive (only reason would be a progress bar/dialog).

How would you suggest to post the request after starting the local event loop? Because as soon as I'm calling exec(), the execution is blocked there (sorry, I think that I missed something in your suggestion).
Actually, I'm also thinking about an alternative solution with processEvents() + a timeout, but I'm not sure if it's the right way to go.

Thanks again to everyone for your help!

J.Hilk

@SimonSchroeder said in Never ending local Event Loop:

I'm very close to suggesting to use a solution with processEvents()...

QEventLoop internally also only calls ProcessEvents but with some reasonable flags set. It's just only marginally better than calling that function by hand.

Maybe I'm over-estimating the cost, but actually I have many calls that were originally corresponding to the application code being called synchronously through functions. Now, many of these calls have been replaced to calls to micro-services, and it seemed easier at that time to simply mimic the synchronous behavior of functions by making these requests synchronous too.

Actually, I'm also thinking about an alternative solution with processEvents() + a timeout, but I'm not sure if it's the right way to go.

It's not.

I'ts not meant to be run in this way, I myself also have run in strange and unexplainable behaviour in the past. Solved itself when I Learned to Stop Worrying and Love the Asynch.

How would you suggest to post the request after starting the local event loop? Because as soon as I'm calling exec(), the execution is blocked there (sorry, I think that I missed something in your suggestion).

QNetworkAccessmanager also has a finished signal. Connect to that before you make your request that one should always be called, even if it returns immediately.

https://doc.qt.io/qt-6/qnetworkaccessmanager.html#finished

Ekia

Actually, there is one thing that I still don't get: how comes that my local event loop can be stuck. I mean, the timer I use should make sure that it is forced to exit after a 10 seconds (even if I somehow miss the signal triggered at the end of the reply), but sometimes, it doesn't happen at all.

SimonSchroeder

@Ekia said in Never ending local Event Loop:

How would you suggest to post the request after starting the local event loop? Because as soon as I'm calling exec(), the execution is blocked there (sorry, I think that I missed something in your suggestion).

You could use QMetaObject::invokeMethod to place something into the local event loop even before it is run. I would usually put the current code into a lambda and place that lambda into the event loop using invokeMethod. (So, the invokeMethod would be done before calling exec().)

@Ekia said in Never ending local Event Loop:

Actually, I'm also thinking about an alternative solution with processEvents() + a timeout, but I'm not sure if it's the right way to go.

In that case I would use QTime (there used to be something like QTime::elapsed(), which in Qt 6 I would now replace with msecsTo(QTime::currentTime())). Usually, processEvents is not a good idea because it will slow your application down. However, if you are already willing to wait up to 10 seconds, maybe it is not too bad. I wish slots could be coroutines so that they are async, but you could write them as if they were synchronous (just with an occasional co_yield).