#648
Issues/containers/gvisor-tap-vsock/#648

gvproxy permanently stops forwarding after brief host network unreachability (WiFi band roam)

Openrjlgopened 6mo ago

Upstream issue draft: gvproxy stops forwarding after brief host-network unavailability

Target repo: containers/gvisor-tap-vsock (cross-reference containers/podman#15095, #18017)

Submission: review and file via gh issue create --repo containers/gvisor-tap-vsock --web --title "..." --body-file docs/notes/gvproxy-wifi-roam-issue.md (skipping the heading lines), or paste into the GitHub web UI.

Title

gvproxy permanently stops forwarding after brief host network unreachability (WiFi band roam)

Body

Summary

On macOS Podman (applehv provider), a brief period of host-network unavailability — in the observed case, a 2.5-minute WiFi band roam from 5 GHz back to 2.4 GHz and back — causes gvproxy to log a burst of "network is unreachable" errors and then stop forwarding permanently. The host network recovers seconds later, but gvproxy never does. From that moment on, every host→VM API call fails (socket refused) and every published port is dead, while podman machine list continues to report Running because vfkit is still alive. Manual podman machine stop && podman machine start is the only way out.

Environment

  • macOS 26.4.1 (Apple Silicon, Mac16,9)
  • Podman 5.8.2 (Homebrew)
  • Provider: applehv (vfkit)
  • gvproxy: v0.8.7
  • WiFi (no Ethernet)

Reproduction (observational, hard to force on demand)

The trigger is any event where the host loses its default route or LAN reachability for 1–3 minutes while the VM continues to run. WiFi roams between bands appear to be sufficient. I have not been able to force-reproduce reliably; the failure was reconstructed from logs.

Evidence — minimal log excerpt

/var/folders/.../T/podman/gvproxy.log (last lines before gvproxy goes silent forever):

time="2026-04-24T23:24:36+10:00" level=error msg="Can't proxy a datagram to udp: write udp 192.168.8.177:50125-><external>:51820: write: can't assign requested address"
time="2026-04-24T23:24:40+10:00" level=error msg="Can't proxy a datagram to udp: dial udp <external>:51820: connect: network is unreachable"
... (~25 similar lines over 2 min 11 s) ...
time="2026-04-24T23:26:47+10:00" level=error msg="Can't proxy a datagram to udp: dial udp <external>:51820: connect: network is unreachable"
<EOF — no further entries>

Concurrent macOS log (log show --predicate 'subsystem == "com.apple.WirelessRadioManager.Coex:Public"'):

2026-04-24 23:24:33  WiFiS: ASSOCIATED_5G ch60 -> ASSOCIATED_2G ch1
2026-04-24 23:24:35  isRemoteServerReachable = NO  (associated, no path)
2026-04-24 23:26:42  WiFiS: ASSOCIATED_2G ch1 -> ASSOCIATED_5G ch60
2026-04-24 23:26:50  isRemoteServerReachable = YES

So host network was unreachable from 23:24:35 to 23:26:50 (~2 min 15 s). gvproxy stops logging at 23:26:47 — three seconds before the host recovers — and never logs again, even though its process is still alive. Host→VM forwarding is broken until podman machine stop && start.

Expected behaviour

gvproxy should treat transient network is unreachable errors on outbound dials as recoverable: continue running and resume forwarding once the host network returns. At minimum, listeners on host ports should not be torn down, and the API socket forward should remain functional regardless of whether the VM has working internet egress.

Actual behaviour

After a window of network is unreachable errors, gvproxy enters a state where it stops emitting log output and stops forwarding host↔VM traffic, but the process remains alive. Recovery requires podman machine stop && start.

Workaround

We run a launchd-driven watchdog that detects the wedged state (podman machine list says Running but the API socket refuses) and restarts the machine. Reference implementation: <link to your repo's scripts/bin/homelab-podman-watchdog.ts once published>. This is a workaround we'd like to retire — hence this issue.

Related

  • containers/podman#15095 — "forwarded ports stop responding after network load" (open)
  • containers/podman#18017 — "mac version of gvproxy runs for a while and stops forwarding" (closed without fix)
  • containers/gvisor-tap-vsock#584 — "100% CPU usage on macOS on sleep" (open) — possibly related host-state mishandling

Asks

  1. Can gvproxy be made to retry/skip transient outbound dial failures without entering a permanent silent state?
  2. If gvproxy detects a sustained host-network outage, should it tear down and rebuild listeners on recovery rather than wedging?
  3. Is there an existing diagnostic flag that would have made this easier to root-cause from the host side?

Activity

4
alexbakker5mo ago

Just ran into this as well.

The root cause appears to be that the UDP proxy creates a connected socket on the host via net.Dial("udp", ...) / connect(). The kernel binds it to the source IP of the interface that the route to the destination goes through at that moment. When the host's network changes (VPN reconnect, WiFi roam), that source IP is removed from the interface, but the socket remains. Every subsequent write() fails with EADDRNOTAVAIL ("can't assign requested address"), which is logged but doesn't trigger socket recreation.

This is reproducible with any long-lived UDP flow. For example: I have a somewhat strange setup with a WireGuard tunnel (PersistentKeepalive enabled) living in a container, that's routed through another VPN tunnel living on the host. The issue occurs when the host VPN tunnel reconnects and gives me a different local IP.

After forcing a new flow (changing WireGuard's listen-port), a new socket is created on the host with the current IP and traffic resumes immediately.

alexbakker5mo ago

Just ran into this as well.

The root cause appears to be that the UDP proxy creates a connected socket on the host via net.Dial("udp", ...) / connect(). The kernel binds it to the source IP of the interface that the route to the destination goes through at that moment. When the host's network changes (VPN reconnect, WiFi roam), that source IP is removed from the interface, but the socket remains. Every subsequent write() fails with EADDRNOTAVAIL ("can't assign requested address"), which is logged but doesn't trigger socket recreation.

This is reproducible with any long-lived UDP flow. For example: I have a somewhat strange setup with a WireGuard tunnel (PersistentKeepalive enabled) living in a container, that's routed through another VPN tunnel living on the host. The issue occurs when the host VPN tunnel reconnects and gives me a different local IP.

After forcing a new flow (changing WireGuard's listen-port), a new socket is created on the host with the current IP and traffic resumes immediately.

llimllib2mo ago

I'm pretty sure that this issue is at the root of my lima/colima setup getting itself wedged following VPN restarts.

When I keep a colima VM (using vz) running for a long time through sleeps and VPN dis/re/connects, I end up with a VM network that can't connect to anything outside the host machine, and only killing the colima usernet network bridge process fixes it. The host machine itself can successfully connect to the network.

If it would be useful and you'd like me to try and get logs for you, let me know, I could try to reproduce.

Versions:

  • lima 2.2.0 (which vendors gvisor-tap-vsock 0.8.9)
  • colima 0.10.3
fraschm199818d ago

Confirming on a newer stack. Still reproduces with the gvproxy from podman 6.1.2 (macOS 26 / Darwin 25.5, Apple silicon, applehv, machine image 5.2.5). Same setup as @alexbakker: WireGuard inside containers with PersistentKeepalive = 25.

After switching wifi, the old flows stay bound to the previous network's address and never get recreated, while new flows open on the new address:

gvproxy  UDP 10.149.95.198:62984->x.x.x.x:51421   # old wifi, dead
gvproxy  UDP 10.149.95.198:62198->y.y.y.y:51821   # old wifi, dead
gvproxy  UDP 192.168.89.22:62320->x.x.x.x:51421   # after rebind
gvproxy  UDP 192.168.89.22:54130->y.y.y.y:51821   # after rebind

The keepalive keeps the dead flow from ever reaching the idle timeout, so without intervention the tunnel stays down indefinitely.

Workaround, in case it helps others: a launchd agent waits on com.apple.system.config.network_change (e.g. notifyutil -w) and runs wg set wg0 listen-port 0 in each container. The new source port makes gvproxy create a fresh socket and traffic resumes within seconds. It's a workaround, though. Re-dialing the host socket when a write returns EADDRNOTAVAIL would fix it for any long-lived UDP flow.