Summary
On a TSI (virtio-vsock) guest, an unconnected SOCK_DGRAM socket that talks to more than one peer gets
the wrong source address from recvfrom()/recvmsg(): the kernel returns the destination of the most
recent sendto() on that socket, whoever actually sent the datagram. Protocols that multiplex peers on
one UDP socket and match replies by source address (for example OpenAFS Rx) break or misattribute replies.
Evidence
- Guest kernel, libkrunfw (
libkrun/libkrunfw)patches/0011-Transparent-Socket-Impersonation-implementation.patch,tsi_dgram_recvmsg(): after the vsockrecvmsg,memcpy(msg->msg_name, tsk->sendto_addr, tsk->sendto_addr_len).tsk->sendto_addris overwritten by everytsi_dgram_sendmsg()with amsg_name. - Host VMM, libkrun
src/devices/src/virtio/vsock/tsi_dgram/unix.rs,recv_to_pkt(): usesrecv(), so the real sender is dropped before the datagram reaches the guest (no field for it in the VSOCK_OP_RW packet).TsiDgramProxy.sendto_addris likewise a single address per proxy. - Reproduction: the script below (stdlib only; also at https://forgejo.mit.edu/djbclark/get/src/branch/main/docs/experiments/udp-source-test.py), two DNS servers A and B:
Run on a Hyades microVM (libkrun TSI, guest kernel 6.12.109) on 2026-10-16. A single-peer socket is fine.
A then B: reply to query 1 (sent to 18.0.70.160) reported from 18.0.72.3 BAD A then B: reply to query 2 (sent to 18.0.72.3) reported from 18.0.72.3 ok B then A: reply to query 3 (sent to 18.0.72.3) reported from 18.0.70.160 BAD - Consequence seen: OpenAFS
afsd.fuse(one Rx socket for the VLDB on 7003 and fileservers on 7000) mounts butls /afs/athena.mit.edunever completes; replies are matched to the wrong peer and the requests are retransmitted forever. - No existing issue found (searched libkrun and libkrunfw issues and PRs for recvfrom, source address,
sendto_addr, tsi_dgram_recvmsg; Jina, Parallel and Exa web search). Related but different: libkrunfw#132
(
tsi_connectcommits to S_INET too early with SOCK_DGRAM), PR 659 (sendto_addr bind).
Expected / fix direction
The VMM knows the true sender (recvfrom() on the host socket). Carry it in the datagram to the guest
(e.g. a small sockaddr header before the payload, gated by a feature bit so old kernels still work) and
have tsi_dgram_recvmsg copy that into msg_name, instead of tsk->sendto_addr. Until then document that
multi-peer UDP on one socket is unsupported under TSI.
Workarounds
- One connected socket per peer in the application (works: the kernel's
getpeername/connected path is exact for a single peer). - virtio-net with passt/gvproxy instead of TSI (not available on Hyades).
Reproduction script
import socket
import struct
import sys
A = (sys.argv[1] if len(sys.argv) > 1 else "18.0.70.160", 53)
B = (sys.argv[2] if len(sys.argv) > 2 else "18.0.72.3", 53)
def query(qid):
return (struct.pack(">HHHHHH", qid, 0x0100, 1, 0, 0, 0)
+ b"\x07example\x03com\x00" + struct.pack(">HH", 1, 1))
def run(label, seq):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.settimeout(4)
for qid, dst in seq:
s.sendto(query(qid), dst)
sent = {qid: dst[0] for qid, dst in seq}
for _ in seq:
data, src = s.recvfrom(1500)
qid = struct.unpack(">H", data[:2])[0]
ok = "ok " if src[0] == sent[qid] else "BAD"
print(f"{label}: reply to query {qid} (sent to {sent[qid]}) reported from {src[0]} {ok}")
run("A then B", [(1, A), (2, B)])
run("B then A", [(3, B), (4, A)])Found while trying to run the OpenAFS FUSE client in a libkrun guest (Hyades, MIT SIPB). Happy to test or contribute a fix if the direction above (carry the real sender in the datagram, gated by a feature bit) is acceptable.
Activity
0No activity yet.