Qt Installer Framework 7zip exception not enough memory

[lbaudouin]

Hi,

When I want to generate repository using repogen on Windows10 with Qt Installer Framework 3.0.
I have an issue with 7zip:

repogen.exe -p packages repository
Caught 7zip exception: Cannot create archive "<full_path>\1.0.3lib.7z": not enough memory

I have at least 24GB of free memory, and enough disk space. Manual 7zip works perfectly.

Note: The lib folder contains some .lib and one of them is big: 387MB (total 410MB).

Is there any limitation of QtInstallerFramework? How can I use big files with this tool?

[koahnig]

@lbaudouin

Hi and welcome to devnet forum

I doubt that there is a limitation, but I do not have enough insight there.

Try to remove the big library in order to check, if it is the real problem.

[JonB]

@lbaudouin
There's nothing "big" about 400MB as far as 7Zip is concerned! Sounds like something else is wrong.

[lbaudouin]

If I replace the 400MB by a smaller one (100MB), it works.

[koahnig]

@lbaudouin

As @JonB I consider this as odd. I am using 7z on a regular basis with really large files.

The only explanation possible is that there is part of 7z integrated in the installer and that part is creating the problem.

This is merely a user driven forum. Typically with Installer FrameWork questions not a lot of feedback is anticipated. Possibly on the developer mailing list there are some of IFW developers around.
They would know also if there are really such limitations.

[AdamC1228]

I too am running into this issue. However I am trying to build the offline installer. When use the release build for my program the libraries are small enough that they do not trigger this issue. Using the Debug build causes the binarycreator to throw a fit. In my case it really does not like Qt5Cored.dll. When that dll is present it triggers the following:

[2757] Warning: QFile::remove: Empty or null file name
Caught exception: Cannot create archive "C:\Users\XXXX\AppData\Local\Temp\binarycreator-l9Mfi9\com.XXXX.xxxxx\2.0.0content.7z": not enough memory

I'm not sure if this helps matters at all but if you find out anything from the developer mailing list please drop a comment here.

BTW, I am using the 3.0.4 release for windows.

[mselizabeth005]

This post is deleted!

[miller9163286]

This post is deleted!

[kuro]

Hi,
I'm facing just same problem.
After some investigation, I found thant this is caused by multithread build of 7zip codes.
Though I have not solved this bug(must be a bug) yet, I found we can use Single Threaded IFW by specifing _7ZIP_ST preprocessor macro on building IFW.
(If you can make effort to build IFW by yourself)
anyway, I'll make bug report to IFW team.

[aha_1980]

@kuro

Thanks for reporting this. Please add a link to the report her once you've done it, so others can follow.

Regards

[kuro]

O key, I see, I'll put link to the issue after submit it.
because I'm now doing some more investigation(for writing more precise report), it will takes more time.
Thank you for your guide.

[kuro]

I searched bug tracker site for this issue, then found this:
https://bugreports.qt.io/browse/QTIFW-1157
This issue is already closed because this is duplication of this:
https://bugreports.qt.io/browse/QTIFW-957

Regards.

[koahnig]

@kuro said in Qt Installer Framework 7zip exception not enough memory:

I searched bug tracker site for this issue, then found this:
https://bugreports.qt.io/browse/QTIFW-1157
This issue is already closed because this is duplication of this:
https://bugreports.qt.io/browse/QTIFW-957

Regards.

Thanks for following up and reporting links

[aha_1980]

@kuro Thanks for your findings. You can vote and comment at the bug QTIFW-957 to give it more priority :)

[dcouch]

@kuro Where are you defining this preprocessor macro?

[KMKT]

Hi! This topic is pretty old; but the problem remains...
I am currently using IFW 4.2 with a Core i9 processor with10 cores => 20 logical processors.
I have 32GB RAM but I guess that the 32-bit official version 4.2 can only handle 2GB...
I bet the high number of threads combined with limited 32-bit memory is a bad combination.
Building the IFW in 64-bit (or, as kuro suggests: limiting the number of threads) would help.
However, many Qt users (like me) just want to use the official build installed by the MaintenanceTool.
Can we expect the forthcoming official release version 4.3 to fix this? I see a recent git commit about using libarchive for 7zip. The comment seems to tell that using libarchive will not be official but will require a special build option...

[Christian Ehrlicher]

@KMKT said in Qt Installer Framework 7zip exception not enough memory:

Can we expect the forthcoming official release version 4.3 to fix this?

Wrong place to ask such things, this is a user-driven forum -> bugreports.qt.io

[KMKT]

@Christian-Ehrlicher Oh! I'm sorry.
My first search on bugreports.qt.io only found closed topics that did not fix the problem, and I was too shy to open a new one while so many others were closed. Then I found this open topic on this forum.
Anyway, after a second search on bugreports.qt.io, I found this: https://bugreports.qt.io/browse/QTIFW-2375
This will probably fix the problem. I guess, we'll have to wait for v4.3.0...
In the meantime, I can use my old 4-core computer to build installers; or use zip compression instead of 7zip, which results in a 10% larger file size (+50MB in my case).

[Stephen Quan]

@KMKT writes:

In the meantime, I can use my old 4-core computer to build installers; or use zip compression instead of 7zip, which results in a 10% larger file size (+50MB in my case).

I have been having this issue for 3 years now, but, your sentence gave me a new perspective on this issue. For me, the computer that always fails has 12 cores, whereas the computer that always works has 4 cores.

I decided to do an experiment on my failing 12 core machine: I run MSCONFIG, go to Boot options go into Advanced Options I tried changing my cores to either 4 or 8. And, oh my gosh, to my surprise, in both tests, the problem went away?!?! So, there must be some fault in the 7zip library not handling a large number of cores?!

[KMKT]

@Stephen-Quan Thank you for detailing how to limit the number of processors. I did not know it was possible.
Remark: my Intel Core i9 CPU has 10 cores, which makes 20 logical processors. So if I set the number of processors to 8, it iactually limits my CPU to 4 cores.