Would it be possible to add a TCPTimeWaitTimeout field in types.Configuration?
Currently, the 60s timeout can cause long stalls and tank connections. There's an upstream fix, but the PR hasn't got reviewed yet.
Details from Claude below:
gvisor-tap-vsock accepts all guest TCP through tcp.NewForwarder (services.go#L28-L30). When the host side closes first, gVisor's endpoint sits in TIME_WAIT for its default 60 s. The guest closed second, so it holds no TIME_WAIT and Linux soon reuses the source port for the same destination. gVisor reopens a TIME_WAIT 4-tuple only for a listening endpoint, and a forwarder stack has none, so the new SYN and all its retransmissions are dropped silently (google/gvisor#15013; fix google/gvisor#15019, open, unreviewed). The guest's connect() hangs for 7–68 s.
Measured through Lima 2.1.3 on macOS (vz, in-process gvisor-tap-vsock v0.8.9, Fedora 44 guest):
| guest→host workload, host closes first | stock | 1 s TIME_WAIT (patch below) |
|---|---|---|
| 8 workers × 500 connections | all 8 stall 68 s | all 8 stall 1.04 s |
| 4 workers × 5,000, unthrottled | 48.6 conn/s, 24 stalls of ~68 s | 2,171 conn/s, 24 stalls of ~1.05 s |
| 1 worker at 10 conn/s, 5 min | 12 stalls, three of ~68 s | 0 connects > 1 s, max 24 ms |
ForwardMaxInFlightDrop stayed 0 throughout, so this is not #676. v0.9.0 creates the stack the same way (createStack), with no TCP options set.
The patch used for the right-hand column, in createStack:
tw := tcpip.TCPTimeWaitTimeoutOption(time.Second)
if err := s.SetTransportProtocolOption(tcp.ProtocolNumber, &tw); err != nil {
return nil, errors.New(err.String())
}Ask: a TCPTimeWaitTimeout field in types.Configuration beside v0.9.0's TCPMaxInFlight / TCPConnectTimeout (configuration.go#L63-L67), applied as above, or a short default. The link is a local virtual NIC, so late duplicates from an old connection aren't a practical risk, and when gVisor closed first the guest's own state doesn't depend on gVisor's TIME_WAIT. A short timer only shrinks the stall to the guest's next SYN retransmission (~1 s, as the table shows). google/gvisor#15019 removes it: a Lima build with it backported had 0 connects over 1 s in 35,000 connections. So a gVisor bump once #15019 merges is the real fix, and the option is a stopgap until then.
Assisted-by: Claude (Claude Code)
Activity
0No activity yet.