Possible thread stack size problem on macOS with deep recursion

[Volker_75]

Possible thread stack size problem on macOS with deep recursion

Hello,

I am the co-developer of FET, a Qt-based timetabling application, and I am investigating an intermittent crash reported by a macOS user.

The application performs timetable generation in several worker threads. Currently these threads are created using std::thread, for example:

std::thread([t]{
    timetablingThreads[t].startGenerating();
}).detach();

The generation algorithm contains a large recursive function, Generate::randomSwap(). The recursion depth is explicitly limited to 14.

The user reported that with multiple generation threads the application sometimes crashes on macOS, while single-thread generation seems stable. The original macOS crash report showed memory allocation functions such as QArrayData::allocate() / malloc(), which initially made me suspect heap corruption.

To investigate this I built FET with AddressSanitizer:

-O1
-g
-fsanitize=address
-fno-omit-frame-pointer
-fno-optimize-sibling-calls

and linked with:

-fsanitize=address

ASan is definitely active.

With ASan, the problem becomes reproducible almost immediately. ASan reports:

ERROR: AddressSanitizer: stack-overflow

The worker-thread stack reported by ASan is approximately:

size 0x83000

so roughly 524 KiB.

The stack trace contains many recursive calls like:

Generate::randomSwap(int, int) generate.cpp:38619
Generate::randomSwap(int, int) generate.cpp:38619
...

until ASan reports the stack overflow.

As a test, I replaced the std::thread creation with QThread::create() and explicitly set a larger stack:

QThread* thread = QThread::create([t]{
    timetablingThreads[t].startGenerating();
});

thread->setStackSize(16 * 1024 * 1024);

connect(thread, &QThread::finished,
        thread, &QObject::deleteLater);

thread->start();

With this change, the immediate ASan stack overflow disappears and generation continues normally for much longer.

So my questions are:

1. Does this look like a normal consequence of the relatively small default worker-thread stack on macOS, especially with ASan and a large recursive function?
2. Would explicitly increasing the thread stack size be the appropriate permanent fix for this kind of application?
3. Is there anything Qt-specific I should consider when using QThread::setStackSize() on macOS?
4. Could the original non-ASan crash, which appeared inside malloc() / QArrayData::allocate(), plausibly have been another manifestation of stack exhaustion, or should I continue looking for an independent heap corruption/data race?
5. Is 16 MiB a reasonable stack size here, or would you recommend another approach?

Environment:

• macOS on Apple Silicon
• Qt 6.11.x
• Clang / Xcode
• C++17
• FET timetable generator
• up to 4 generation threads in the reported case
• recursion depth limited to 14

I would appreciate any advice on whether increasing the worker-thread stack is the correct solution, or whether there is something else I should investigate before making this change permanent.

[IgKh]

Welcome to the forum!

@Volker_75 said in Possible thread stack size problem on macOS with deep recursion:

so roughly 524 KiB.

Yes, this tracks. The default stack size for threads created via the pthreads APIs in macOS is 512 KB, which is way smaller than on other Unix-like systems (e.g. on Linux secondary stacks are 8 MB by default). This is not the only way in which Apple's pthreads implementation is different. I don't like it all that much.

I think that in macOS distributing jobs to a pool is typically done with the GCD (Grand Central Dispatch) API, but I suppose you don't really want to use non-portable APIs.

I'm not aware of any particular specific issues with increasing the stack size up from the default. Non specific concerns are the usual ones: allocating unbounded amount of stack data is generally frowned upon, and large stacks can exhaust address space (not really a problem if the number of threads is static). So it is really up to your own design constraints and preferences.

[Volker_75]

Thank you for answering. I already tried to measure a lot and investigate. The strange thing is, that the memory usage under Linux looks much smaller (measures with "valgrind -v --tool=massif --stacks=yes ./fet" and discussing the issue with AI is difficult. chat-gpt sadly refuse to answer several of my questions because of "cyber security issues". So if you (other guys) have more idea, please let me know.

[IgKh]

@Volker_75 said in Possible thread stack size problem on macOS with deep recursion:

chat-gpt sadly refuse to answer several of my questions because of "cyber security issues".

Pretty funny, but I can see why it might trigger the guardrails. Stack smashing attacks (e.g. trying to purposefully overflow the stack) are a crude but common type of attack. You can perhaps reframe the conversation to analyzing why the macOS version consumes more stack space than the Linux version - if it is indeed what you think is happening.

I don't think valgrind is very reliable on macOS though? Its hard to see through the black magic going inside the Objective C runtime (and make no mistake, even if your application's code is 100% C++, you ARE using objective C behind the scenes). You should try to use the Xcode instruments to measure memory usage.

If you can make a minimal reproducing example that overflows the stack on macOS but not on Linux (with the same stack size), post it and I can try to debug it as well.

[SGaist]

Hi,

I agree with @IgKh about the safety of changing the stack size. It's also part of using the system as designed. You simply require more of it and can request so.
You can find more information in the Apple developer documentation.

[SimonSchroeder]

On the one hand you are saying that you have restricted the recursion depth to 14 levels, but on the other hand you are saying you have a stack overflow. This does not make sense unless you have really huge stack frames (any arrays on the stack?). Or your actual recursion depths is much deeper than 14 levels.

It is quite likely that the stack overflow is the real problem and malloc doesn't have to do anything with it. For now, I would stop looking elsewhere.

Setting a larger stack size certainly seems to help. Somebody mentioned that Linux stack size is 8MB, so you could even just go with 8MB (instead of 16MB) if your software works on Linux. However, this is not a permanent fix! Over time people will run larger and larger problems. You'll eventually run out of stack space on all operating systems, not just macOS. The best solution might be to rewrite your recursive algorithm with a loop. (This is technically also not a permanent solution, but the only restriction is not enough RAM and there is nothing you can do about it in software.)