WoW64 thread registry fills after 512 cumulative threads, causing wrong TEB fallback and eventual crash

WoW64 thread registry fills after 512 cumulative threads, causing wrong TEB fallback and eventual crash

Environment

• Madeira: v0.1.3 (4)
• Device: iPhone 14 Pro (iPhone15,2)
• iOS: 27.0 (24A437)
• increased-memory-limit: enabled
• JIT: Built-in StikJIT + LocalDevVPN
• Guest application: 32-bit x86 / WoW64
• Game engine: BGI / BURIKO General Interpreter

Problem

When running a 32-bit x86 application through WoW64, Madeira's Mach exception thread registry appears to become permanently exhausted after 512 cumulative thread registrations.

This does not seem to correspond to 512 simultaneously live threads.

Shortly before exhaustion, the log shows only a few dozen live FEX lookup-cache states, while cumulative thread creation is already at 511:

[lookup-cache] ml606 mode=l1-only alloc=2MB live=23 cumulative=511 fleet_live=46MB

The final available registry entry is then used:

Mach exception handler registered thread ... (idx=511)

Immediately afterward, newly-created threads can no longer be registered:

[thread-registry] FULL (512 slots) — thread ... NOT registered; Mach events on it will resolve to the slot-0 TEB (wrong process!)

After this point, the application may continue running temporarily, but eventually exception handling appears to use an invalid or incorrect thread context.

Resulting crash

In one reproduction, execution eventually reached a region without execute permission:

[fault_rip] ... kr=2(PROTECTION_FAILURE) ...
[mach_exc] UNHANDLED ...

At that point, the guest thread state was clearly invalid:

State.RIP=0x0
RSP=0x23
live_rsp=0x23 out of range

Wine then failed to dispatch the exception because the exception frame was outside the valid stack range:

call_seh_handlers invalid frame ...
NtRaiseException Exception frame is not in stack limits => unable to dispatch exception.
NtTerminateProcess(...)

The Windows/Wine process is then terminated and Madeira returns to the library UI.

Reproduction

1. Launch a 32-bit x86 Windows application through Madeira/WoW64.
2. Use an application that repeatedly creates and destroys worker threads.
3. Let it run until cumulative thread creation approaches 512.
4. Observe the last successful registry entry:

(idx=511)

5. The next newly-created threads begin reporting:

[thread-registry] FULL (512 slots)

6. Continue running the application.
7. Eventually an exception is delivered with an invalid/wrong thread context and the WoW64 process terminates.

On my latest reproduction, the 512-slot limit was reached within only a few minutes.

Expected behavior

Registry entries belonging to terminated threads should be removed or reused.

The registry should not become permanently exhausted after 512 cumulative thread creations when only a few dozen thread states are currently live.

Actual behavior

The registry appears to retain entries across thread termination until all 512 slots are consumed.

After the limit is reached, Madeira explicitly reports that new Mach exception events will fall back to the slot-0 TEB:

Mach events on it will resolve to the slot-0 TEB (wrong process!)

This is followed later by invalid exception delivery, corrupted/unusable guest thread state, and termination of the WoW64 process.

Possible cause

The Mach-thread-to-TEB registry may not be unregistering or reusing entries when guest/FEX threads exit.

Simply increasing the registry size would likely only delay the problem.

A proper fix may be to remove the corresponding Mach-thread-to-TEB mapping when a thread terminates, or otherwise safely reuse freed registry slots.

Additional notes

This does not appear to be a JIT startup problem:

• JIT enables successfully.
• CS_DEBUGGED becomes SET.
• The built-in debugger detaches successfully.
• Wine/WoW64 starts normally.
• The 4 GB WoW guest window is reserved successfully.

It also does not appear to be an out-of-memory / jetsam termination in the reproduced crash.

log.txt