QTcpSocket data transfer when read buffer is full, but does not resume when it frees up

[edmundas919]

I have server-client application where client sends data packets to server and server reads them at set intervals. Client sends data faster than server can read and that uses all the memory on the server. I've used QTcpSocket::setReadBufferSize() to limit buffer size on server side and now when servers read buffer is full, data is buffered on client side, which is what i want, but when read buffer frees up, data transfer does not resume.
I have downloaded Qt(5.10.0) sources and pdb's and i can see that QAbstractSocket::readData(), where d->socketEngine->setReadNotificationEnabled(true) line is, never enters, because in QIODevicePrivate::read() it is entered only when buffer is empty (qiodevice.cpp line 1176). In my case read buffer never gets empty, because i read only complete packets and if bytes available is not enough for complete packet i leave it in QTcpSocket.
Shouldn't QAbstractSocket::readData() be called when read buffer is not full anymore and not when it's completely empty? Or maybe i do something wrong?

[VRonin]

Could you simply receive the data continously on the server side,write it to file and process it periodically by reading the file?

[edmundas919]

That just shifts from one bottleneck (RAM) to another bottleneck (HDD) and when client sends gigabytes of data it's a bad idea. And in case server crashes, someone will need to manage those large files and re-sending that data will occupy network again which is wasting resources.
The best solution is for client to see if server is busy and don't send data and I found workaround how to do that.
In Qt5.10 sources i can clearly see that QTcpSpcket internal read notifications is disabled (qabstractsocket.cpp; bool QAbstractSocketPrivate::canReadNotification(); line 697) when read buffer is full and to enable read notifications you need to read all buffer to make it empty OR use QAbstractSocket::setReadBufferSize(newSize) which internally enables read notifications WHEN newSize is not 0 (unlimited) and not equal to oldSize (qabstractsocket.cpp; void QAbstractSocket::setReadBufferSize(qint64 size); line 2824). Here's a short function for that:

QTcpSocket socket;
qint64 readBufferSize; // Current max read buffer size.
bool flag = false; // flag for changing max read buffer size.
bool isReadBufferLimitReached = false;

void App::CheckReadBufferLimitReached()
{
    if (readBufferSize <= socket.bytesAvailable())
        isReadBufferLimitReached = true;
    else if (isReadBufferLimitReached)
    {
        if (flag)
        {
            readBufferSize++;
            flag = !flag;
        }
        else
        {
            readBufferSize--;
            flag = !flag;
        }
        socket.setReadBufferSize(readBufferSize);
        isReadBufferLimitReached = false;
    }
}

In the function which reads data from QTcpSocket at the set intervals, BEFORE reading data, I call this function, which checks if read buffer is full and sets isReadBufferLimitReached if true. Then I read needed amount of data from QTcpSocket and AT THE END I call that function again, which, if buffer were full before, calls QAbstractSocket::setReadBufferSize(newSize) to set new buffer size and enable internal read notifications. Changing read buffer size by +/-1 should be safe, because you read at least 1 byte from socket.

And on client side you can use QAbstractSocket::bytesToWrite(), QAbstractSocket::flush() or QAbstractSocket::waitForBytesWritten() to know if server is busy.