[MSVC/Qt6] Address Space Exhaustion during "Edit and Continue" after migrating from Qt3 to Qt6.8.3

DevMattew

Hi everyone,

I am currently managing a massive migration of a legacy industrial system from Qt 3 (C++14) to Qt 6.8.3 (C++17).
The architecture consists of 5 core central libraries distributed across more than 100 executable binaries. Everything is built using Visual Studio 2019 (.vcxproj).

The Issue:
In our legacy Qt 3 environment, the "Edit and Continue" (Hot Reload) functionality worked perfectly across the entire suite. However, after migrating to Qt 6.8.3, we are facing a critical failure.

The application crashes after only 2 or 3 recompilations/iterations. The error reported is an insufficient reserved address space (Address Space Exhaustion). It seems that each "Edit and Continue" cycle is leaking or fragmenting the virtual address space to the point where the OS can no longer allocate the necessary contiguous memory for the new binary delta.

Observations:

This was not an issue in Qt 3.
The crash is consistent: it happens after a few iterations of code changes and hot-swapping.
We are seeing this across multiple executables, suggesting a systemic change in how memory is handled between the two versions.
Questions:

Does anyone know if Qt 6 has introduced new mechanisms (e.g., larger metadata tables, different alignment requirements, or changes in QObject memory layouts) that could impact the memory overhead during a hot-patching session?
Are there specific MSVC compiler flags or linker options (related to /incremental or /LARGEADDRESSAWARE) that are now mandatory when combining Qt 6 with Edit and Continue?
Has anyone experienced "Address Space Exhaustion" specifically during hot-reloading in a large-scale Qt 6 project?
Any leads or technical insights into the internal mechanism of how Qt 6 interacts with the MSVC debugger during Edit and Continue would be greatly appreciated.

Environment:

IDE: Visual Studio 2019
Framework: Qt 6.8.3
Standard: C++17
OS: Windows

JKSH

@DevMattew said in [MSVC/Qt6] Address Space Exhaustion during "Edit and Continue" after migrating from Qt3 to Qt6.8.3:

Does anyone know if Qt 6 has introduced new mechanisms (e.g., larger metadata tables, different alignment requirements, or changes in QObject memory layouts) that could impact the memory overhead during a hot-patching session?
Are there specific MSVC compiler flags or linker options (related to /incremental or /LARGEADDRESSAWARE) that are now mandatory when combining Qt 6 with Edit and Continue?
Has anyone experienced "Address Space Exhaustion" specifically during hot-reloading in a large-scale Qt 6 project?

First, can you please provide more details on how this "hot-swapping"/"hot-patching"/"hot-reloading" is implemented and how it works? This doesn't sound like native Qt functionality.

AFAICS, there is official support for hot-reloading in QML, but not in Qt C++.

DevMattew

Hi, First thanks for your response.
I am not referring to a Qt-specific API or a framework feature. I am talking about the MSVC 'Edit and Continue' feature provided by the Visual Studio debugger.

In our workflow, we modify C++ code during a debug session, and the MSVC debugger applies the changes to the running process. This worked seamlessly with our Qt 3 binaries. However, with Qt 6.8.3, the MSVC debugger seems to exhaust the process's reserved address space after a few iterations, leading to a crash.

My concern is that the increased binary size or the different memory layout of Qt 6 (compared to Qt 3) might be triggering this address space exhaustion within the MSVC debugger's patching mechanism.

SimonSchroeder

On Windows I now constantly monitor the "committed" memory. I still might have lots of free RAM, but committed memory also includes paged memory. My computer tends to slow down when the committed memory is full. With MSVC I only had some errors related to out of heap space during compilation. Yours might be related to that. Did you try to restart your computer before a debugging session? Only this will free up committed memory.

DevMattew

Hi @SimonSchroeder,
Thanks for the tip. I did try restarting the computer before a session, but unfortunately, the issue persists.

To give a more precise update on the behavior of this bug:

I have 5 core central libraries. I can recompile and use Edit and Continue on these libraries as many times as I want without any crashes. This suggests that the basic integration of Qt 6 and MSVC is stable for standard module sizes.

The crash occurs exclusively when I modify the main source files of my executables.

If the file is small or empty, 'Edit and Continue' works fine.
When the file is large (specifically those reaching 10,000+ lines), the application crashes after only 2 or 3 iterations.
Since I am in a pure x64 environment, I am convinced this is not a global memory shortage, but rather a failure in the MSVC debugger's ability to handle the patching of massive translation units. It seems that the larger binary footprint of Qt 6/C++17, combined with these oversized files, is triggering a critical fragmentation of the reserved address space during the injection of the binary delta.

Has anyone seen this 'size-dependent' crash with 'Edit and Continue' in large-scale Qt 6 projects?

JKSH

@DevMattew said in [MSVC/Qt6] Address Space Exhaustion during "Edit and Continue" after migrating from Qt3 to Qt6.8.3:

I am not referring to a Qt-specific API or a framework feature. I am talking about the MSVC 'Edit and Continue' feature provided by the Visual Studio debugger.

Ah, I see. I have no experience with this functionality, so I'll defer to those who do.

If you don't get any clues here, try posting at the Interest mailing list where Qt engineers are active (you'll need to subscribe first): https://lists.qt-project.org/listinfo/interest

DevMattew

Hi @JKSH, thanks a lot

hskoglund

Perhaps upgrading from MSVC 2019 to MSVC 2022 could help with the crashes (just a guess)

SimonSchroeder

Depending on your exact use case you might want to have a look into other solutions for hotpatching, like https://liveplusplus.tech/

DevMattew

Hi, quick update after further investigation.

I upgraded from Visual Studio 2019 to Visual Studio 2022 (Thanks to @hskoglund), and this gave me much more detailed Edit and Continue diagnostics.
These logs made the issue much clearer.

What I found
This does not appear to be a Qt runtime memory leak.
The bottleneck is in the MSVC Edit-and-Continue/hot reload patching workload when large executable translation units are edited.

For the same type of edit in source.cpp:

Legacy setup (Qt3-era project):

~1,542 sections processed
~1 MB remapped
patch apply time ~0.3 s

Qt6.8.3 setup (VS2022, x64, C++17):

~11,050 sections processed
~67 MB remapped
hundreds of modified/new sections (including many unwind/PDATA-related sections)
patch apply time ~25 s
So the slowdown/crash pattern is strongly correlated with:

very large source files in executable entry modules, and
much larger ENC delta/metadata in modern Qt6/C++17 builds.
Important detail
I can Edit-and-Continue my central libraries repeatedly without issue.
The failures happen mostly when modifying very large executable source files (10k+ lines).

Current conclusion
This looks like a size-dependent MSVC ENC limitation/fragility (address-space fragmentation/exhaustion during repeated remapping), rather than a Qt API feature issue.

If anyone has practical mitigation experience for large x64 Qt6 projects (compiler/linker/debug settings, ENC-friendly code organization, etc.), I’m very interested.

Thanks again for all suggestions.

nicholas_ru

is for me, i am read instructios, when porting from qt5 to qt6. If you have qt3, have script for it.But i am doing this is not in Windows.

DevMattew

Quick update: I finally found a workaround!

Huge thanks to @SimonSchroeder for mentioning Live++. While I didn't adopt the tool itself, I looked into their recommended compilation flags to minimize PDB size.

By applying specific flags in my build config files to reduce the number of linked sections, I managed to bring the section count back down. This completely stopped the address space exhaustion and the crashes during "Edit and Continue."

The Trade-off:
It increases the executable size slightly and, more importantly, disables exception handling (any throw/catch will now cause a crash). As long as we avoid try/catch blocks in these large files, it works perfectly.

Problem solved! Thanks for the help everyone.