Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Memory leak in QNetworkProxy (should i post on bugtracker?)
QtWS25 Last Chance

Memory leak in QNetworkProxy (should i post on bugtracker?)

Scheduled Pinned Locked Moved Unsolved General and Desktop
qnetworkreplyqnetworkaccessmqnetworkrequestqnetworkpoxymemory leak
3 Posts 2 Posters 1.5k Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • V Offline
    V Offline
    Vlad_Savelyev
    wrote on 8 Jul 2016, 18:24 last edited by Vlad_Savelyev 7 Aug 2016, 18:40
    #1

    I working on a web-spider application that uses lot of proxies. The problem is the memory leaks are enormouse, up to 100 mb in 5 minutes. I wrote a simple test like this, which runs inside QThread:

    worker::process()
    {
    //here i read the proxies from file first
      QString readAll;
      QFile file("proxies.txt");
      if(!file.open(QIODevice::ReadWrite))
      {
         qDebug() << file.errorString();
         emit finished();
         eturn;
      }
      QTextStream in(&file);
    
          while(!in.atEnd())
          {
    
              readAll = in.readAll();
    
          }
          file.close();
    
       
    //split the proxies by new line to the QStringList
      QStringList lst = readAll.split(QRegExp("(\\r\\n)|(\\n\\r)|\\r|\\n"), QString::SkipEmptyParts);
    
      foreach(QString proxy,lst){
    
            QNetworkRequest request;
            request.setUrl(QUrl("https://google.com/"));
    
            request.setRawHeader("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/2022021 Firefox/46.0");
    
            QStringList parts = proxy.split(":");
            QString s_port = parts[1];
          // here is the thing
            QNetworkProxy webProxy;
                webProxy.setHostName(parts[0]);
                webProxy.setPort(s_port.toUShort());
                webProxy.setType(QNetworkProxy::Socks5Proxy);
    
               m.setProxy(webProxy);
    
    
               QNetworkReply * reply = m.get(request);
               QTimer timer;
               timer.setSingleShot(true);
    
               QEventLoop loop;
               QObject::connect(&timer, SIGNAL(timeout()), &loop, SLOT(quit()));
               QObject::connect(reply, SIGNAL(finished()), &loop, SLOT(quit()));
               timer.start(1*1000);
               loop.exec();
    
               if(reply->isFinished())
                   reply->abort();
    
               reply->close();
               delete reply;
         }
    }
    

    So in this foreach im doing the requests, and see memory goes SUPER crazy, and it never decreases even if i stop the thread. But if i comment out this:

    
     QNetworkProxy webProxy;
           /*     webProxy.setHostName(parts[0]);
                webProxy.setPort(s_port.toUShort());
                webProxy.setType(QNetworkProxy::Socks5Proxy);*/
    
               m.setProxy(webProxy);
    

    So basically no proxy request, i dont see any leakages. All is stable. Is it a possible bug? I don't know if i write post here or report to bugtracker? Also im using a small timeout with QEventLoop here, its just for the sake of example, but in reality, i need to abort the request if the proxy is invalid, otherwise it could hang for hours (maybe even days)

    K 1 Reply Last reply 8 Jul 2016, 19:14
    0
    • V Vlad_Savelyev
      8 Jul 2016, 18:24

      I working on a web-spider application that uses lot of proxies. The problem is the memory leaks are enormouse, up to 100 mb in 5 minutes. I wrote a simple test like this, which runs inside QThread:

      worker::process()
      {
      //here i read the proxies from file first
        QString readAll;
        QFile file("proxies.txt");
        if(!file.open(QIODevice::ReadWrite))
        {
           qDebug() << file.errorString();
           emit finished();
           eturn;
        }
        QTextStream in(&file);
      
            while(!in.atEnd())
            {
      
                readAll = in.readAll();
      
            }
            file.close();
      
         
      //split the proxies by new line to the QStringList
        QStringList lst = readAll.split(QRegExp("(\\r\\n)|(\\n\\r)|\\r|\\n"), QString::SkipEmptyParts);
      
        foreach(QString proxy,lst){
      
              QNetworkRequest request;
              request.setUrl(QUrl("https://google.com/"));
      
              request.setRawHeader("User-Agent","Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/2022021 Firefox/46.0");
      
              QStringList parts = proxy.split(":");
              QString s_port = parts[1];
            // here is the thing
              QNetworkProxy webProxy;
                  webProxy.setHostName(parts[0]);
                  webProxy.setPort(s_port.toUShort());
                  webProxy.setType(QNetworkProxy::Socks5Proxy);
      
                 m.setProxy(webProxy);
      
      
                 QNetworkReply * reply = m.get(request);
                 QTimer timer;
                 timer.setSingleShot(true);
      
                 QEventLoop loop;
                 QObject::connect(&timer, SIGNAL(timeout()), &loop, SLOT(quit()));
                 QObject::connect(reply, SIGNAL(finished()), &loop, SLOT(quit()));
                 timer.start(1*1000);
                 loop.exec();
      
                 if(reply->isFinished())
                     reply->abort();
      
                 reply->close();
                 delete reply;
           }
      }
      

      So in this foreach im doing the requests, and see memory goes SUPER crazy, and it never decreases even if i stop the thread. But if i comment out this:

      
       QNetworkProxy webProxy;
             /*     webProxy.setHostName(parts[0]);
                  webProxy.setPort(s_port.toUShort());
                  webProxy.setType(QNetworkProxy::Socks5Proxy);*/
      
                 m.setProxy(webProxy);
      

      So basically no proxy request, i dont see any leakages. All is stable. Is it a possible bug? I don't know if i write post here or report to bugtracker? Also im using a small timeout with QEventLoop here, its just for the sake of example, but in reality, i need to abort the request if the proxy is invalid, otherwise it could hang for hours (maybe even days)

      K Offline
      K Offline
      kshegunov
      Moderators
      wrote on 8 Jul 2016, 19:14 last edited by
      #2

      @Vlad_Savelyev
      Hello,
      Could you prepare a minimal working example that reproduces this behavior. I could test on my machine, and you'd probably be asked for it anyway if you report as a bug.

      Kind regards.

      Read and abide by the Qt Code of Conduct

      1 Reply Last reply
      0
      • V Offline
        V Offline
        Vlad_Savelyev
        wrote on 8 Jul 2016, 19:53 last edited by
        #3

        sure i will reply with the project code

        1 Reply Last reply
        0

        1/3

        8 Jul 2016, 18:24

        • Login

        • Login or register to search.
        1 out of 3
        • First post
          1/3
          Last post
        0
        • Categories
        • Recent
        • Tags
        • Popular
        • Users
        • Groups
        • Search
        • Get Qt Extensions
        • Unsolved