Environment
- libkrun 1.19.4 (Arch package
libkrun 1.19.4-1), libkrunfw 5.5.0-1, crun-based krun 1.29.1-1 - Host: Arch Linux x86_64, kernel 6.12.91, KVM; rootless podman 5.x with pasta
- Guest:
python:3.12-slim(bookworm, glibc 2.36) - Networking: default — no NIC, so implicit TSI with
TsiFlags::HIJACK_INET(tsi_hijackon the guest kernel command line) - The guest-side code involved is libkrunfw's kernel patch
net/tsi/af_tsi.c, which is byte-identical between the v5.5.0 tag and current libkrunfw main — filing here (not libkrunfw) following the precedent of #511.
Summary
The guest TSI accept() path holds lock_sock(sk) on the listening socket
while blocking on the vsock round-trip to the VMM. Any concurrent
getsockname() on the same listening socket therefore blocks forever.
On non-TSI Linux, accept() does not hold the socket lock across blocking,
which is why the spawn-accept-thread / getsockname pattern (standard in
debugpy's serve(), and in many servers) works everywhere else.
This reproduces on a fixed-port listener with no inbound connection at
all — merely having one thread blocked in accept() while another thread
calls getsockname() on the same socket deadlocks. It is independent of the
ephemeral-port divergence reported in #818 .
Minimal repro (tested: host finishes instantly, krun guest hangs)
cat > d.py <<'EOF'
import socket, threading, time
s = socket.socket(); s.bind(('0.0.0.0', 45678)); s.listen(5)
t = threading.Thread(target=s.accept, daemon=True) # blocks in guest TSI accept
t.start()
time.sleep(0.5)
print('calling getsockname...', flush=True)
print('getsockname:', s.getsockname(), flush=True) # never returns under TSI
print('SCRIPT DONE', flush=True)
EOF
python3 d.py # host sanity: prints everything and exits
podman run --rm --runtime=krun \
--annotation krun.ram_mib=1024 --annotation krun.cpus=2 \
--mount type=bind,src=$PWD/d.py,target=/d.py \
python:3.12-slim timeout 15 python3 /d.pyHost output:
calling getsockname...
getsockname: ('0.0.0.0', 45678)
SCRIPT DONEkrun guest output (killed by timeout 15; would hang forever):
calling getsockname...Real-world capture
Seen while debugging why debugpy hangs inside a krun container
(daemonized adapter process, strace -f):
81 ... bind(4, {AF_INET, 0.0.0.0:5678}, 16) = 0
81 ... listen(4, 4096) = 0
81 ... clone3(...) = 82 <- debugpy serve() spawns its accept thread
82 ... (accept path, blocks on the vsock round-trip to the VMM)
81 ... getsockname(4, <unfinished ...> <- never returnsRelation to #511
Same defect class as #511 ("severe lock contention when creating/destroying connections" — 3 conn/s with 90% loss under TSI vs 2747–5747 conn/s with pasta/passt). This report shows the contention is not only a performance issue: it can deadlock unrelated syscalls operating on the same socket.
Suggested fix direction
Don't hold sk_lock across the blocking VMM wait — e.g. release/reacquire
around the round-trip, or use a separate lock for proxy state.
Activity
2Same lock, wider reach (measured on SIPB's Hyades runner, libkrun "2.0-sipb", guest kernel 6.12.91; details in #926):
- A blocking
recv()/read()holds it as well asaccept(), and the calls it blocks includesend(),shutdown()andgetpeername(), not onlygetsockname().duplex-probe.c: a thread parked inrecv()on a connected socket and another threadsend()ing on it: the data reaches the peer only after therecv()is released (normal kernel: 0 to 1 ms). - A thread parked in
poll()does not hold the lock, so from userspace one can wait inpoll()before a blockingread/accept; an LD_PRELOAD shim doing that made a 1,417-test Gradle/okhttp suite on that runner go from 7 failures per run to 1 (the remaining one also fails elsewhere). - A listener closed while another thread is blocked in
accept()is not released; a new listener on the same port then loses most connections: #929.
Follow-up: same investigation, same result with a real NIC — with
--annotation krun.use_passt=1thed.pyrepro printsgetsocknameand exits normally, so the deadlock is TSI-specific. Consistent with the pasta/passt contrast in #511's benchmarks.