netstack: close() and shutdown(SHUT_WR) wait out the 200ms TCP_CORK timer instead of flushing

Open

Description

When a TCP socket has TCP_CORK set and holds less than a full segment, close() and shutdown(SHUT_WR) don't send that data. The held data and the FIN both wait for the cork timer (#10140), so the peer gets them ~200ms late. Clearing TCP_CORK before closing sends them right away.

On Linux, closing or shutting down a corked socket sends pending data and the FIN immediately. From tcp(7): "If set, don't send out partial frames. All queued partial frames are sent when the option is cleared again." Linux also pushes queued data when it sends a FIN.

This affects stock nginx. With sendfile on; tcp_nopush on;, which is Fedora's packaged config and common elsewhere, nginx corks each response. On a connection without keep-alive, it closes the socket without uncorking first. Under runsc, every such request takes ~200ms. An nginx/ab benchmark showed this as a 57× gap against runc (205ms vs 3.6ms per request at -c 100). The gap drops to ~13× with tcp_nopush off.

Here's nginx serving one request under runsc --strace:

nginx X setsockopt(0x5 socket:[9], SOL_TCP, TCP_CORK, ...) = 0
nginx X writev(0x5 socket:[9], ..., 0x1) = 235
nginx X sendfile(0x5 socket:[9], 0xe /usr/share/fedora-testpage/index.html, ...) = 8484
nginx X close(0x5 socket:[9]) = 0                        # 15:09:39.373803
ab    X close(0x4 socket:[8]) = 0                        # 15:09:39.575456, ~201.7ms later

In pkg/tcpip/transport/tcp/snd.go, a partial data segment under cork starts the timer and returns without sending. The merge loop stops at the zero-payload FIN segment queued behind it, and nothing in this path takes the pending FIN into account. So the FIN waits behind the held segment until corkTimerExpired drains it: https://github.com/google/gvisor/blob/4eb0c8ea2a33950153a8cfcc14d08df6efcbd099/pkg/tcpip/transport/tcp/snd.go#L871-L911

This is present in release-20260921.0 and on master as of 4eb0c8ea. #15926 / #16122 changed this block but only cover full segments.

Steps to reproduce

cork_close.py: a loopback server that sends 100 bytes per connection and closes, and a client that times connect-to-EOF:

import socket, sys, threading, time

def serve(srv, mode):
    while True:
        conn, _ = srv.accept()
        if mode != "plain":
            conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_CORK, 1)
        conn.sendall(b"x" * 100)
        if mode == "cork-uncork-close":
            conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_CORK, 0)
        if mode == "cork-shutdown":
            conn.shutdown(socket.SHUT_WR)
        conn.close()

for mode in ["plain", "cork-close", "cork-uncork-close", "cork-shutdown"]:
    srv = socket.socket()
    srv.bind(("127.0.0.1", 0))
    srv.listen(128)
    threading.Thread(target=serve, args=(srv, mode), daemon=True).start()
    times = []
    for _ in range(20):
        start = time.monotonic()
        c = socket.create_connection(srv.getsockname())
        while c.recv(4096):
            pass
        times.append((time.monotonic() - start) * 1000)
        c.close()
    times.sort()
    print(f"{mode:18} median {times[10]:8.2f} ms   max {times[-1]:8.2f} ms")

docker run --rm --runtime runc  -v $PWD:/r python:3.13-slim python3 /r/cork_close.py
docker run --rm --runtime runsc -v $PWD:/r python:3.13-slim python3 /r/cork_close.py

runc:

plain              median     0.04 ms   max    35.55 ms
cork-close         median     0.04 ms   max     0.13 ms
cork-uncork-close  median     0.04 ms   max     0.19 ms
cork-shutdown      median     0.03 ms   max     0.13 ms

runsc:

plain              median     0.35 ms   max    46.39 ms
cork-close         median   202.51 ms   max   203.83 ms
cork-uncork-close  median     0.36 ms   max     1.16 ms
cork-shutdown      median   202.55 ms   max   203.36 ms

With nginx, Fedora's packaged config shows the same thing, even with no concurrency:

docker run -d --runtime runsc --name web fedora:latest sh -c 'dnf install -y nginx httpd-tools && nginx && sleep infinity'
docker exec web ab -n 200 -c 1 http://127.0.0.1/      # ~203ms per request; ~0.08ms under runc

Setting tcp_nopush off; (or sendfile off;) brings it to ~0.55ms.

runsc version

runsc version release-20260921.0
spec: 1.2.1

Default platform (systrap). --platform=kvm behaves the same: 205.8ms per request on the nginx benchmark.

docker version (if using docker)

28.5.1

This ran in privileged Docker-in-Docker on a Fedora 44 host. The nginx numbers reproduce on bare metal: containerd CRI with the runsc shim on an AWS m5.metal gives 205ms per request at -c 100.

uname

Linux 7.2.9-201.fc44.x86_64 #1 SMP PREEMPT_DYNAMIC Fri Oct 9 15:04:46 UTC 2026 x86_64

Activity

No activity yet.