JSON file transfer from Rpi to PC

Andrex_Qt

Hello,

I have created a small program that will make a JSON text file "form1.JSON" On RPI. The file has around 100 Objects.
Now I want to send the data from that file to PC via TCP protocol.
Both PC and RPI are on same network.
at RPI i use following code:

void MainWindow::ToTheServer()
{
    QString jstr;
    QJsonDocument doc1;
    QByteArray ba;

   QFile j("/home/pi/form1.JSON");
    if(j.exists())
    {
        if(j.open(QIODevice:: ReadOnly| QIODevice::Text))
        {
                jstr=j.readAll();
                doc1= QJsonDocument::fromJson(jstr.toUtf8());
                ba=doc1.toJson();
                socket->putChar(101);
                socket->write(ba);
                ui->label_2->setNum(ba.size());
        }
        j.close();

        socket->putChar(105);
        socket->close();
    }
}
void MainWindow::on_Btotheserver_clicked()
    {
        ToTheServer();
    }

And at PC

server::server(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::server)
{
    ui->setupUi(this);
    Ser = new QTcpServer(this);
    socket = new QTcpSocket(this);
    connect(Ser, SIGNAL(newConnection()), this, SLOT(newConnection()));
    if(!Ser->listen(QHostAddress::AnyIPv4, 1234))
        ui->label->setText( "Err!");
    times = new  QTimer(this);
    connect(times,SIGNAL (timeout()),this,SLOT(fromRaspberry()));
    times->start(2);
}
void server::newConnection()
{
    socket = Ser->nextPendingConnection();
}
void server::fromRaspberry()
{
    QString q;
    if(socket->isOpen())
    {
            socket->getChar(&variable);
            if(variable==101)
            {
                    q=socket->readAll());
                    ui->textBrowser_2->append(q);
                    socket->getChar(&variable2);
                    ui->label->setNum(variable2);
                    if(variable2==105)
                              socket->close();
            }
    }
    else 
    {
        variable=1;
        variable2=1;
    }
}

after the connection is established
through button press event I send the data from RPI to PC (who is continuously listening). Data reception complete.
The issue here is the data received here is truncated most of the time. on some occasions i get all the 100 objects and some garbage data.
Please suggest a solution.
Thank you for reading.

JonB

@Andrex_Qt
I don't know why you're using some QTimer.

In a word, you cannot assume that q=socket->readAll()); will read a complete message: it will only read what happens to be there at the instant it is called. And you are not buffering anything. And you don't have any transactions. And you can't do readAll() followed by getChar().

I suggest you look at the Qt TCP/IP client example (Fortune?), and considerably rewrite your client approach correspondingly. Yours does not look good to me.

Andrex_Qt

@JonB
ok,

Question: is the sender part OK? the first code at RPI side.
At client side you are suggesting that one shouldn't use socket->readAll(). I tested DataStream functions also getChar() (on Client side in place of socket->readAll() ) & also putChar() at RPI side results were same.
I have gone through the Fortune server and Fortune Client examples did not understand them so I implemented simple TCP Server-Client using QTcpServer and QTcpSocket.

JonB

@Andrex_Qt said in JSON file transfer from Rpi to PC:
Sender is probably OK. Sending code is easier than receiving code.

Though having said that, why your client goes "open-send-close" I don't know. If it later has more to send, it's more usual to open the connection once beforehand and close it later on, keeping the same socket open to the server the whole time.

And your proposed sending protocol won't work, at least not easily. Your one character at the start is OK, but your "terminating" one character after the JSON is going to problematic to recognise at the server side.

At client side you are suggesting that one shouldn't use socket->readAll().

I certainly never said that.

I have gone through the Fortune server and Fortune Client examples did not understand them

Then that is why your code does not work and does not take the right approach. I suggest you try, try again till you understand the principles, don't know what else to say.

Hint: no QTimer at server side. Either look at https://doc.qt.io/qt-5/qiodevice.html#startTransaction / https://doc.qt.io/qt-5/qdatastream.html#startTransaction, or use https://doc.qt.io/qt-5/qiodevice.html#readyRead as the signal and be prepared to accumulate/buffer the bytes received.

There is also https://wiki.qt.io/WIP-How_to_create_a_simple_chat_application example, I don't know if that helps you.

Andrex_Qt

@JonB
sorry for late reply. you suggestion worked well.
i followed https://wiki.qt.io/WIP-How_to_create_a_simple_chat_application and modified my code according to that. now at sender side looks like this

QJsonDocument doc1;
QFile j("/home/first.JSON"); 
if(j.exists())
        {
            if(j.open(QIODevice:: ReadOnly| QIODevice::Text))
            {
                jstr=j.readAll();
                doc1= QJsonDocument::fromJson(jstr.toUtf8());
            }
            j.close();

            QDataStream clientStream(socket);
            clientStream.setVersion(QDataStream::Qt_5_7);
            // Create the JSON we want to send
            QJsonObject message;
            message[QStringLiteral("type")] = QStringLiteral("message");
            clientStream << QJsonDocument(doc1).toJson();
        }
    }
    else {
        ui->l_err->show();
        ui->l_err->setText("Connection refused. Please check server.");
    }

and used server application from the link above at receiver end.
this works well. I have tested this with JSON file having 3000 object. Will this work for 50,000 samples and more possibly 1500000 objects?