Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. QTemporaryFile takes forever
Forum Update on Tuesday, May 27th 2025

QTemporaryFile takes forever

Scheduled Pinned Locked Moved Solved General and Desktop
qtemporaryfile
12 Posts 5 Posters 2.9k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • J Offline
    J Offline
    jsulm
    Lifetime Qt Champion
    wrote on 26 Oct 2015, 10:33 last edited by
    #2

    So, you want to store each object in its own file?
    And you have "very big number of objects", right?
    Then you will create a very big number of temporary files, I don't think such an approach will deliver a good performance.

    https://forum.qt.io/topic/113070/qt-code-of-conduct

    1 Reply Last reply
    1
    • V Offline
      V Offline
      VRonin
      wrote on 26 Oct 2015, 11:30 last edited by
      #3

      That's not exactly the point, if I simulate the same behaviour using QFile the results is MUCH faster and creating the file does not depend on the number of files already in the folder

           QList<QFile*> allfiles;
           QTime timeCounter;
           QString tempFileName;
           QFile file("C:/Temp/ChangeAverageTime.txt");
           file.open(QIODevice::WriteOnly | QIODevice::Text);
           QTextStream stream(&file);
           timeCounter.start();
           for (int i = 0; i < 2000; ++i) {
               if (i % 100 == 0) {
                   stream << QString("%1 Files. Time: %2").arg(i).arg(static_cast<double>(timeCounter.restart()) / 1000.0, 0, 'f', 2) << '\n';
               }
               tempFileName.clear();
               do { // Generate random file name
                   for (int j = 0; j < 15; ++j) {
                       tempFileName.append(static_cast<char>(97 + (qrand() % 26)));
                   }
               } while (QFile::exists("C:/temp/TempTest/" + tempFileName));
               allfiles.append(new QFile("C:/temp/TempTest/" + tempFileName));
               allfiles.last()->open(QIODevice::WriteOnly);
           }
           file.close();
           while (!allfiles.isEmpty()) {
               auto tempDel = allfiles.takeLast();
               tempDel->remove();
               delete tempDel;
           }
           return 0;
      

      0 Files. Time: 0.00
      100 Files. Time: 0.03
      200 Files. Time: 0.03
      300 Files. Time: 0.03
      400 Files. Time: 0.03
      500 Files. Time: 0.03
      600 Files. Time: 0.03
      700 Files. Time: 0.03
      800 Files. Time: 0.03
      900 Files. Time: 0.03
      1000 Files. Time: 0.03
      1100 Files. Time: 0.03
      1200 Files. Time: 0.03
      1300 Files. Time: 0.03
      1400 Files. Time: 0.03
      1500 Files. Time: 0.03
      1600 Files. Time: 0.03
      1700 Files. Time: 0.03
      1800 Files. Time: 0.03
      1900 Files. Time: 0.03

      "La mort n'est rien, mais vivre vaincu et sans gloire, c'est mourir tous les jours"
      ~Napoleon Bonaparte

      On a crusade to banish setIndexWidget() from the holy land of Qt

      J 1 Reply Last reply 16 Dec 2019, 17:52
      0
      • J Offline
        J Offline
        jsulm
        Lifetime Qt Champion
        wrote on 26 Oct 2015, 13:13 last edited by
        #4

        I think creating temporary files is slower compared to creating new files, because Qt framework has to ensure that the file names are unique.

        https://forum.qt.io/topic/113070/qt-code-of-conduct

        1 Reply Last reply
        1
        • V Offline
          V Offline
          VRonin
          wrote on 26 Oct 2015, 14:39 last edited by
          #5

          I'm doing that too in: while (QFile::exists("C:/temp/TempTest/" + tempFileName)); but it's lightning fast

          "La mort n'est rien, mais vivre vaincu et sans gloire, c'est mourir tous les jours"
          ~Napoleon Bonaparte

          On a crusade to banish setIndexWidget() from the holy land of Qt

          1 Reply Last reply
          0
          • J Offline
            J Offline
            jsulm
            Lifetime Qt Champion
            wrote on 27 Oct 2015, 06:46 last edited by
            #6

            Why shouldn't it be fast? You only check whether a file exists or not. What QTemporaryFile is doing: it creates a unique file name and ensures nobody else creates one with the same name in the same directory. This, for sure, involves some OS calls to make the creation of a temp file an atomic operation. Else several processes could create and use a temp file with same file name.

            https://forum.qt.io/topic/113070/qt-code-of-conduct

            1 Reply Last reply
            3
            • M Offline
              M Offline
              Mecanica
              wrote on 16 Dec 2019, 14:39 last edited by
              #7

              I know this thread is too old, but I would resolve this by using an inmemory database like sqlite, where you can have two columns one for the file name an another for the file data stored as a QByteArray.

              J 1 Reply Last reply 16 Dec 2019, 17:28
              0
              • M Mecanica
                16 Dec 2019, 14:39

                I know this thread is too old, but I would resolve this by using an inmemory database like sqlite, where you can have two columns one for the file name an another for the file data stored as a QByteArray.

                J Offline
                J Offline
                JonB
                wrote on 16 Dec 2019, 17:28 last edited by
                #8

                @Mecanica
                Whatever the merits of your suggestion, that would not be a suitable replacement for a QTemporaryFile method in the Qt framework!

                1 Reply Last reply
                1
                • V VRonin
                  26 Oct 2015, 11:30

                  That's not exactly the point, if I simulate the same behaviour using QFile the results is MUCH faster and creating the file does not depend on the number of files already in the folder

                       QList<QFile*> allfiles;
                       QTime timeCounter;
                       QString tempFileName;
                       QFile file("C:/Temp/ChangeAverageTime.txt");
                       file.open(QIODevice::WriteOnly | QIODevice::Text);
                       QTextStream stream(&file);
                       timeCounter.start();
                       for (int i = 0; i < 2000; ++i) {
                           if (i % 100 == 0) {
                               stream << QString("%1 Files. Time: %2").arg(i).arg(static_cast<double>(timeCounter.restart()) / 1000.0, 0, 'f', 2) << '\n';
                           }
                           tempFileName.clear();
                           do { // Generate random file name
                               for (int j = 0; j < 15; ++j) {
                                   tempFileName.append(static_cast<char>(97 + (qrand() % 26)));
                               }
                           } while (QFile::exists("C:/temp/TempTest/" + tempFileName));
                           allfiles.append(new QFile("C:/temp/TempTest/" + tempFileName));
                           allfiles.last()->open(QIODevice::WriteOnly);
                       }
                       file.close();
                       while (!allfiles.isEmpty()) {
                           auto tempDel = allfiles.takeLast();
                           tempDel->remove();
                           delete tempDel;
                       }
                       return 0;
                  

                  0 Files. Time: 0.00
                  100 Files. Time: 0.03
                  200 Files. Time: 0.03
                  300 Files. Time: 0.03
                  400 Files. Time: 0.03
                  500 Files. Time: 0.03
                  600 Files. Time: 0.03
                  700 Files. Time: 0.03
                  800 Files. Time: 0.03
                  900 Files. Time: 0.03
                  1000 Files. Time: 0.03
                  1100 Files. Time: 0.03
                  1200 Files. Time: 0.03
                  1300 Files. Time: 0.03
                  1400 Files. Time: 0.03
                  1500 Files. Time: 0.03
                  1600 Files. Time: 0.03
                  1700 Files. Time: 0.03
                  1800 Files. Time: 0.03
                  1900 Files. Time: 0.03

                  J Offline
                  J Offline
                  JonB
                  wrote on 16 Dec 2019, 17:52 last edited by
                  #9

                  @VRonin said in QTemporaryFile takes forever:

                       } while (QFile::exists("C:/temp/TempTest/" + tempFileName));
                       allfiles.append(new QFile("C:/temp/TempTest/" + tempFileName));
                       allfiles.last()->open(QIODevice::WriteOnly);
                  

                  @VRonin I see that you wrote this in 2015. Leaving aside that if the timing you describe still holds then I am not sure why and yours is an interesting finding, looking at this code now would you recommend it for exclusively creating a file, in a multi-process environment? ;-)

                  Christian EhrlicherC 1 Reply Last reply 16 Dec 2019, 19:01
                  0
                  • J JonB
                    16 Dec 2019, 17:52

                    @VRonin said in QTemporaryFile takes forever:

                         } while (QFile::exists("C:/temp/TempTest/" + tempFileName));
                         allfiles.append(new QFile("C:/temp/TempTest/" + tempFileName));
                         allfiles.last()->open(QIODevice::WriteOnly);
                    

                    @VRonin I see that you wrote this in 2015. Leaving aside that if the timing you describe still holds then I am not sure why and yours is an interesting finding, looking at this code now would you recommend it for exclusively creating a file, in a multi-process environment? ;-)

                    Christian EhrlicherC Offline
                    Christian EhrlicherC Offline
                    Christian Ehrlicher
                    Lifetime Qt Champion
                    wrote on 16 Dec 2019, 19:01 last edited by
                    #10

                    @JonB The code above is not correct and can create a race condition since another thread/process can create exactly the same filename between QFile::exists() and open(). That's also the reason why the Qt implementation is slower - it opens a file and if it fails it checks some error conditions.
                    Also the possible space for the file names is lower in QTemporaryFile (only six digits by default) while the other loop is using 15 characters, so the chance of a collision is much lower.

                    Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
                    Visit the Qt Academy at https://academy.qt.io/catalog

                    J 1 Reply Last reply 16 Dec 2019, 23:55
                    2
                    • Christian EhrlicherC Christian Ehrlicher
                      16 Dec 2019, 19:01

                      @JonB The code above is not correct and can create a race condition since another thread/process can create exactly the same filename between QFile::exists() and open(). That's also the reason why the Qt implementation is slower - it opens a file and if it fails it checks some error conditions.
                      Also the possible space for the file names is lower in QTemporaryFile (only six digits by default) while the other loop is using 15 characters, so the chance of a collision is much lower.

                      J Offline
                      J Offline
                      JonB
                      wrote on 16 Dec 2019, 23:55 last edited by
                      #11

                      @Christian-Ehrlicher
                      It was the race condition that I had noticed. I was bringing that to @VRonin 's attention and seeing what he said about it, it's sometimes interesting to look back on your own code from years ago :)

                      1 Reply Last reply
                      0
                      • V Offline
                        V Offline
                        VRonin
                        wrote on 17 Dec 2019, 16:46 last edited by
                        #12

                        The bug was solved in Qt 5.8 so a while ago. I'll mark the thread as solved as well

                        "La mort n'est rien, mais vivre vaincu et sans gloire, c'est mourir tous les jours"
                        ~Napoleon Bonaparte

                        On a crusade to banish setIndexWidget() from the holy land of Qt

                        1 Reply Last reply
                        0

                        • Login

                        • Login or register to search.
                        • First post
                          Last post
                        0
                        • Categories
                        • Recent
                        • Tags
                        • Popular
                        • Users
                        • Groups
                        • Search
                        • Get Qt Extensions
                        • Unsolved