Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. QJsonDocument::fromJson fails on "foreign" characters
Forum Updated to NodeBB v4.3 + New Features

QJsonDocument::fromJson fails on "foreign" characters

Scheduled Pinned Locked Moved Unsolved General and Desktop
qjsondocumentutf8fromjson
17 Posts 4 Posters 10.7k Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • V Offline
    V Offline
    VRonin
    wrote on 17 Aug 2016, 15:24 last edited by VRonin
    #8

    This works for me. Qt 5.5.1 on MSVC2013

    #include <QtCore/QCoreApplication>
    #include <QtCore/QJsonDocument>
    
    #include <QDebug>
    
    void textMessageReceived(const QString &msg)
    {
        qDebug() << "msg = " << msg << '\n';
        QJsonParseError error;
    
        // Create a Json document from text. Fails for foreign characters!
        QJsonDocument doc = QJsonDocument::fromJson(msg.toUtf8(), &error);
    
        if(doc.isNull())
            qDebug() << "(Broken json), error code = "
                      << error.errorString() << '\n';
        else
            qDebug() << "(Valid json) : " << QString(doc.toJson()) << '\n';
    }
    
    int main()
    {
    
        QString jsontext = "{\"string\": \"abc\"}";
        textMessageReceived(jsontext);
    
        jsontext = QString::fromWCharArray(
                    L"{\"string\": \""
                 L"\u00E6"
                 L"\u00F8"
                    L"\u00E5"
                    L"\"}"
                    );
        textMessageReceived(jsontext);
    
        return 0;
    
    }
    

    Output:

    msg =  "{\"string\": \"abc\"}" 
    
    (Valid json) :  "{\n    \"string\": \"abc\"\n}\n" 
    
    msg =  "{\"string\": \"æøå\"}" 
    
    (Valid json) :  "{\n    \"string\": \"æøå\"\n}\n" 
    

    "La mort n'est rien, mais vivre vaincu et sans gloire, c'est mourir tous les jours"
    ~Napoleon Bonaparte

    On a crusade to banish setIndexWidget() from the holy land of Qt

    P 1 Reply Last reply 17 Aug 2016, 15:38
    5
    • V VRonin
      17 Aug 2016, 15:24

      This works for me. Qt 5.5.1 on MSVC2013

      #include <QtCore/QCoreApplication>
      #include <QtCore/QJsonDocument>
      
      #include <QDebug>
      
      void textMessageReceived(const QString &msg)
      {
          qDebug() << "msg = " << msg << '\n';
          QJsonParseError error;
      
          // Create a Json document from text. Fails for foreign characters!
          QJsonDocument doc = QJsonDocument::fromJson(msg.toUtf8(), &error);
      
          if(doc.isNull())
              qDebug() << "(Broken json), error code = "
                        << error.errorString() << '\n';
          else
              qDebug() << "(Valid json) : " << QString(doc.toJson()) << '\n';
      }
      
      int main()
      {
      
          QString jsontext = "{\"string\": \"abc\"}";
          textMessageReceived(jsontext);
      
          jsontext = QString::fromWCharArray(
                      L"{\"string\": \""
                   L"\u00E6"
                   L"\u00F8"
                      L"\u00E5"
                      L"\"}"
                      );
          textMessageReceived(jsontext);
      
          return 0;
      
      }
      

      Output:

      msg =  "{\"string\": \"abc\"}" 
      
      (Valid json) :  "{\n    \"string\": \"abc\"\n}\n" 
      
      msg =  "{\"string\": \"æøå\"}" 
      
      (Valid json) :  "{\n    \"string\": \"æøå\"\n}\n" 
      
      P Offline
      P Offline
      Per Gunnar Holm
      wrote on 17 Aug 2016, 15:38 last edited by
      #9

      Thanks again @VRonin !

      That means this is platform dependent I suppose. As far as I can see there are no differences between your code and mine (except you use qDebug).
      It would also explain why there hasn't been a torrent of complaints to Qt if this is a bug and not me making a mistake; probably not too many on my platform.
      As stated initially I have Qt 5.6.0 and I'm on Linux (CentOS 6.8).

      1 Reply Last reply
      0
      • V Offline
        V Offline
        VRonin
        wrote on 17 Aug 2016, 15:45 last edited by
        #10

        Even on Qt5.7 works for me.
        2 questions:

        • did you try my code?
        • What encoding do you use on your .cpp file? (in QtCreator you can see it in edit->select encoding

        "La mort n'est rien, mais vivre vaincu et sans gloire, c'est mourir tous les jours"
        ~Napoleon Bonaparte

        On a crusade to banish setIndexWidget() from the holy land of Qt

        P M 2 Replies Last reply 17 Aug 2016, 15:52
        0
        • V VRonin
          17 Aug 2016, 15:45

          Even on Qt5.7 works for me.
          2 questions:

          • did you try my code?
          • What encoding do you use on your .cpp file? (in QtCreator you can see it in edit->select encoding
          P Offline
          P Offline
          Per Gunnar Holm
          wrote on 17 Aug 2016, 15:52 last edited by
          #11

          @VRonin

          Now I did :-)
          Same result (unfortunately):

          msg =  "{\"string\": \"abc\"}" 
          (Valid json) :  "{\n    \"string\": \"abc\"\n}\n" 
          msg =  "{\"string\": \"æøå\"}" 
          (Broken json), error code =  "invalid UTF8 string" 
          

          I checked the encoding as instructed, and the "Text Encoding" window comes up with "UTF-8" high-lighted.
          I am assuming this is OK?
          -- Gunnar

          1 Reply Last reply
          0
          • V VRonin
            17 Aug 2016, 15:45

            Even on Qt5.7 works for me.
            2 questions:

            • did you try my code?
            • What encoding do you use on your .cpp file? (in QtCreator you can see it in edit->select encoding
            M Offline
            M Offline
            mrjj
            Lifetime Qt Champion
            wrote on 18 Aug 2016, 06:15 last edited by mrjj
            #12

            hi
            window 7, Qt 5.7 also shows it ok.

            Linux Mint, Qt 5.5

            Notice it dont render as æøå but still consider it valid.

            P 1 Reply Last reply 18 Aug 2016, 09:35
            1
            • M mrjj
              18 Aug 2016, 06:15

              hi
              window 7, Qt 5.7 also shows it ok.

              Linux Mint, Qt 5.5

              Notice it dont render as æøå but still consider it valid.

              P Offline
              P Offline
              Per Gunnar Holm
              wrote on 18 Aug 2016, 09:35 last edited by
              #13

              Thanks @mrjj
              It looks increasingly as if this is a problem specific to my platform.
              That's good news for the world, but bad news for me, of course, since it decreases the likelihood of getting it fixed :-(

              1 Reply Last reply
              0
              • P Offline
                P Offline
                Per Gunnar Holm
                wrote on 18 Aug 2016, 12:48 last edited by
                #14

                Today I have tested on virtual installations of Ubuntu 16.04 and CentOS-7, both using QT 5.6.1.
                The test program passes without problems there, so this is definitely a CentOS-6 problem.

                M K 2 Replies Last reply 18 Aug 2016, 12:52
                1
                • P Per Gunnar Holm
                  18 Aug 2016, 12:48

                  Today I have tested on virtual installations of Ubuntu 16.04 and CentOS-7, both using QT 5.6.1.
                  The test program passes without problems there, so this is definitely a CentOS-6 problem.

                  M Offline
                  M Offline
                  mrjj
                  Lifetime Qt Champion
                  wrote on 18 Aug 2016, 12:52 last edited by
                  #15

                  @Per-Gunnar-Holm
                  well maybe Qt 5.6.1 has a bug on that distro.

                  I hope Is it an option to use CentOs-7 instead :)

                  1 Reply Last reply
                  0
                  • P Per Gunnar Holm
                    18 Aug 2016, 12:48

                    Today I have tested on virtual installations of Ubuntu 16.04 and CentOS-7, both using QT 5.6.1.
                    The test program passes without problems there, so this is definitely a CentOS-6 problem.

                    K Offline
                    K Offline
                    kshegunov
                    Moderators
                    wrote on 18 Aug 2016, 14:02 last edited by kshegunov
                    #16

                    @Per-Gunnar-Holm

                    QByteArray ba (msg.toStdString().c_str());
                    

                    Bug or no, the above line doesn't seem correct. You should enforce the required encoding (as @VRonin has done) instead of relying on the internal representation of std::string and/or QString.
                    The above should be:

                    QByteArray ba = msg.toUtf8();
                    

                    If you need to output QStrings to the standard streams, attach a QTextStream to them instead of converting the objects to std::string:

                    QTextStream cout(stdout);
                    QTextStream cerr(stderr);
                    QTextStream cin(stdin);
                    

                    Kind regards.

                    Read and abide by the Qt Code of Conduct

                    P 1 Reply Last reply 18 Aug 2016, 14:45
                    2
                    • K kshegunov
                      18 Aug 2016, 14:02

                      @Per-Gunnar-Holm

                      QByteArray ba (msg.toStdString().c_str());
                      

                      Bug or no, the above line doesn't seem correct. You should enforce the required encoding (as @VRonin has done) instead of relying on the internal representation of std::string and/or QString.
                      The above should be:

                      QByteArray ba = msg.toUtf8();
                      

                      If you need to output QStrings to the standard streams, attach a QTextStream to them instead of converting the objects to std::string:

                      QTextStream cout(stdout);
                      QTextStream cerr(stderr);
                      QTextStream cin(stdin);
                      

                      Kind regards.

                      P Offline
                      P Offline
                      Per Gunnar Holm
                      wrote on 18 Aug 2016, 14:45 last edited by
                      #17

                      Thanks @kshegunov !

                      The c_str() was just something we tested along the way!
                      The original code was

                          // Create a Json document from text. Fails for foreign characters!
                          QJsonDocument doc = QJsonDocument::fromJson(msg.toUtf8(), &error);
                      

                      However, all the variations we/I have tried display the same problem (on CentOS 6.5, as we have discovered).

                      1 Reply Last reply
                      0

                      17/17

                      18 Aug 2016, 14:45

                      • Login

                      • Login or register to search.
                      17 out of 17
                      • First post
                        17/17
                        Last post
                      0
                      • Categories
                      • Recent
                      • Tags
                      • Popular
                      • Users
                      • Groups
                      • Search
                      • Get Qt Extensions
                      • Unsolved