QLocalSocket multiple messages

lolopolosko · 24 Jun 2016, 09:43

Hello everyone

I've got 2 applications. One client and the other Server (Windows Service)
Service started and run client automaticly. When client started, it sent message to service that means "I'm running and send me data". Server receive this and send message (about 8-9) to client.
Some messages has more data (10-20 KB) and some little.
But when client received data, it lost some messages

Example:
From service I send next data
One
Two
Three
Four
Five

And client receive
One
Two
Four
Five

I don't know why and how I can fix it...

And after some time I created MessagesQueue.
I post data (In service) into queue and timer (100msec) get 3 messages and send to client
It works now.. But today I reproduce this problem again.. When my local machine freeze (because has a lot of work), I lost some messages.

Can I fix it and how?

raven-worx · wrote on 24 Jun 2016, 09:43

@lolopolosko
You can't expect someone to help you find a mistake in your code, without posting the actual code...

lolopolosko · 24 Jun 2016, 10:19

// CLIENT
void Client::init() {
        m_socket = new QLocalSocket(this);
        m_socket->setReadBufferSize(512000);
        connect(m_socket, SIGNAL(readyRead()), this, SLOT(receivedMessage()));
}

void Client::receivedMessage() {
    QByteArray bytes = m_socket->readAll();
    QDataStream stream(bytes);
    stream.setVersion(QDataStream::Qt_5_5);

    if (m_blockSize == 0) {
        if (bytes.size() < (int)sizeof(quint32)) {
            qDebug() << "Client::receivedMessage: Message is small than quint32";
            return;
        }

        stream >> m_blockSize;
    }

    if (m_block.isEmpty()) {
        m_block = bytes;
    } else {
        m_block = m_block.append(bytes);
    }

    if (m_block.size() < m_blockSize) {
        qDebug() << "Client::receivedMessage: Message is smaller than block size";
        return;
    } else {
        m_blockSize = 0;
        LocalMessage msg = LocalMessage::fromByteArray(m_block, m_socket->socketDescriptor());
        emit message(msg);
        m_block.clear();
    }
}

// SERVICE
void LocalServer::init() {
    m_server = new QLocalServer(this);
    m_server->setSocketOptions(QLocalServer::OtherAccessOption);
}

void LocalServer::postData(LocalMessage& message) {
    QByteArray bytes = message.toByteArray();

    if (m_sockets.contains(message.socketDescription)) {
        if (m_sockets[message.socketDescription]->state() == QLocalSocket::ConnectedState) {
            m_sockets[message.socketDescription]->write(bytes);
            m_sockets[message.socketDescription]->flush();
        } else {
            qDebug() << "LocalServer::postData: Socket cannot connected to the server. Current state of socket" << (int)m_sockets[message.socketDescription]->state();
            emit error("Socket unconected");
        }
    } else {
        qDebug() << "LocalServer::postData: Socket with descriptor" << message.socketDescription << "not found";
        emit error("Not found socket");
    }
}

kshegunov · 24 Jun 2016, 10:42

@lolopolosko
Hello,
Is it possible you don't receive the whole message and it's somehow discarded?
By the way, this:

m_block = m_block.append(bytes);

looks pretty suspicious by itself.

lolopolosko · 24 Jun 2016, 10:52

@kshegunov

m_block = m_block.append(bytes);

I used it because some messages has bigger than 4096 bytes data
And QLocalSocket divided this message on small parts

Is it possible you don't receive the whole message and it's somehow discarded?
Maybe. But how can I know this?

How I can debug QLocalSocket between two processes?

jsulm · wrote on 24 Jun 2016, 10:52

@lolopolosko m_block.append(bytes); is enough no need to assign it to itself

VRonin · 24 Jun 2016, 11:05

you should check for m_socket->bytesAvailable() before calling QByteArray bytes = m_socket->readAll();

The idea is that the packet has it's size as header (as quint32 for example), so check until bytesAvailable is at least the size of the header, read it and wait until bytesAvailable becomes equal to the size you receive

By wait I mean return

lolopolosko · 24 Jun 2016, 11:14

@VRonin said:

you should check for m_socket->bytesAvailable() before calling QByteArray bytes = m_socket->readAll();

The idea is that the packet has it's size as header (as quint32 for example), so check until bytesAvailable is at least the size of the header, read it and wait until bytesAvailable becomes equal to the size you receive

By wait I mean return

See Client::receivedMessage

void Client::receivedMessage() {
// Read all data
    QByteArray bytes = m_socket->readAll();
    QDataStream stream(bytes);
    stream.setVersion(QDataStream::Qt_5_5);

// Read header (quint32)
    if (m_blockSize == 0) {
        if (bytes.size() < (int)sizeof(quint32)) {
            qDebug() << "Client::receivedMessage: Message is small than quint32";
            return;
        }

        stream >> m_blockSize;
    }

// Read data until m_block.size() == m_blockSize
    if (m_block.isEmpty()) {
        m_block = bytes;
    } else {
        m_block = m_block.append(bytes);
    }

    if (m_block.size() < m_blockSize) {
        qDebug() << "Client::receivedMessage: Message is smaller than block size";
        return;
    } else {
        m_blockSize = 0;
        LocalMessage msg = LocalMessage::fromByteArray(m_block, m_socket->socketDescriptor());
        emit message(msg);
        m_block.clear();
    }
}

But I dont use bytesAvailable() (because I use header (m_blockSize))

kshegunov · 24 Jun 2016, 11:18

@lolopolosko
The point is, that this:

if (bytes.size() < (int)sizeof(quint32)) {
    qDebug() << "Client::receivedMessage: Message is small than quint32";
    return;
}

will make you skip a message if you don't receive the integer whole (i.e. the 4 bytes at once). It's somewhat unusual to get so little data, but certainly not impossible. Also casting sizeof doesn't really make any sense. sizeof is not a function, but it's a language construct that will be substituted by the compiler with the proper number when generating the binary ...

lolopolosko · 24 Jun 2016, 11:26

@kshegunov
Yes, I know...

But wait :-)
I redirect qDebug() to a file, and when I read logs, I dont find message "Client::receivedMessage: Message is small than quint32";

kshegunov · 24 Jun 2016, 14:33

@lolopolosko
Well your code looks pretty okay, so if I were you I'd focus my efforts on:

LocalMessage::fromByteArray(m_block, m_socket->socketDescriptor());

There's one thing that also caught my eye, you do realize you're including the header size field (the thing you read in m_blockSize) in the m_block buffer, right?

I still may be missing some subtlety, but currently I don't see anything that's plainly wrong.

Kind regards.

lolopolosko · 24 Jun 2016, 15:30

@kshegunov
Ok.. Now I changed source code for debugging

I send message directly from my service (without queue)
When client started, service send next messages
TypeMessage::Setting1
TypeMessage::Setting2
TypeMessage::Setting3
TypeMessage::Setting4
TypeMessage::Setting5
TypeMessage::Setting6
TypeMessage::Setting7

And client received only next messages:
TypeMessage::Setting1
TypeMessage::Setting2
TypeMessage::Setting4

For Client::receivedMessage method I put at the beginning of a method

qDebug() << "Client::receivedMessage: bytesAvailable" << m_socket->bytesAvailable();

Now I see in log file

17:22:14:457 - Client::receivedMessage: bytesAvailable 5243
17:22:14:457 - ClientController::onSocketMessageReceived: TypeMessage::Setting1
17:22:14:495 - Client::receivedMessage: bytesAvailable 187
17:22:14:495 - ClientController::onSocketMessageReceived: TypeMessage:Setting2
17:22:14:496 - Client::receivedMessage: bytesAvailable 572
17:22:14:496 - ClientController::onSocketMessageReceived: TypeMessage:Setting4

But where is another messages I dont know

If I use message queue in Service, I receive all messages (but sometimes some messages lost)

I think problem in Service method

void LocalServer::postData(LocalMessage& message) {
    QByteArray bytes = message.toByteArray();

    if (m_sockets.contains(message.socketDescription)) {
        if (m_sockets[message.socketDescription]->state() == QLocalSocket::ConnectedState) {
            m_sockets[message.socketDescription]->write(bytes);
            m_sockets[message.socketDescription]->flush();
        } else {
            qDebug() << "LocalServer::postData: Socket cannot connected to the server. Current state of socket" << (int)m_sockets[message.socketDescription]->state();
            emit error("Socket unconected");
        }
    } else {
        qDebug() << "LocalServer::postData: Socket with descriptor" << message.socketDescription << "not found";
        emit error("Not found socket");
    }
}

kshegunov · 25 Jun 2016, 19:41

@lolopolosko
Could you humor me and output both m_blockSize with m_socket->bytesAvailable() side by side?

lolopolosko · wrote on 25 Jun 2016, 19:41

@kshegunov
Hey!! You're right ^_^
But I don't understand.. How it possible??

My new log:

I: 22:36:04:211 - Client::receivedMessage: bytesAvailable 6015
I: 22:36:04:213 - Client::receivedMessage: m_blockSize 6011
I: 22:36:04:216 - ClientController::onSocketMessageReceived: TypeMessage::Setting1
I: 22:36:04:255 - Client::receivedMessage: bytesAvailable 337
I: 22:36:04:257 - Client::receivedMessage: m_blockSize 235
I: 22:36:04:259 - ClientController::onSocketMessageReceived: TypeMessage:Setting2
I: 22:36:04:261 - Client::receivedMessage: bytesAvailable 1008
I: 22:36:04:263 - Client::receivedMessage: m_blockSize 334
I: 22:36:04:265 - ClientController::onSocketMessageReceived: TypeMessage:Setting4

QLocalSocket merge data? And how I can unmerge it?
Now I don't understand how it works... :-(

P.S.
When readyRead is called, the might be more than one message to be read.

lolopolosko · 26 Jun 2016, 19:39

Ok.. I fix it!
@kshegunov Thank you very much!

Now I added \x1C character (INFORMATION SEPARATOR FOUR) at the end message
When client reads data it check if bytesAvailable > m_blockSize and if it true I split data by \x1C character.

kshegunov · wrote on 26 Jun 2016, 19:39

@lolopolosko
Oh, I was just about to suggest you buffer all the received data. But as you've solved your problem then my suggestion is moot. I'm glad it works now.

Kind regards.