#16172
Issues/google/gvisor/#16172

Optimization: Start fdbased recvmmsg with one receive slot and grow it on backlog

OpenEtiennePerotopened 8d ago

The idea

The fdbased recvMMsgDispatcher reads up to MaxMsgsPerRecv (8) packets per recvmmsg, and each slot holds a receive buffer sized for a GSO packet (about 64 KiB). On its first read, the dispatcher fills all 8 slots, and the buffer pools zero every chunk. So every sandbox keeps about 512 KiB of zero-buffers for its whole lifetime, even when they don't use any networking. Even when they do use networking, request/response traffic rarely has more than one packet queued per read, so slots 1-7 never receive data.

The dispatcher could instead start with a single active slot, and pass only the active prefix to recvmmsg. Slots are already filled lazily, so inactive slots hold no buffers. When a read fills every active slot without having to poll first (packets are queuing faster than they are read), the active count would double, and stop doubling at 8. After many consecutive reads that don't fill every active slot (e.g. 256), it would drop back to one slot and release the other slots' buffers. This needs BlockingRecvMMsgUntilStopped to report whether it polled, and the runsc seccomp rule to allow vlen <= MaxMsgsPerRecv instead of vlen == MaxMsgsPerRecv.

Performance impact

  • //test/benchmarks/base:usage_test container memory: -3.3% (Empty), -2.8% (Nginx), -1.0% (Node), p < 0.001.
  • An nginx sandbox uses about 350 KiB less anonymous memory after boot, after small HTTP requests, and after checkpoint/restore.
  • After a bulk download the slots grow to 8, so memory matches today's (by design).
  • Bulk receive throughput and sandbox CPU per GiB show no measurable change in any of 14 iperf3 conditions: CPU quota 0.25/1/none, 1 or 4 streams, GSO on or off, MTU-sized packets.
  • //test/benchmarks iperf, nginx, httpd, redis, node, ruby, startup and size: no metric changes at p < 0.01.

Beat the autoresearch

The following benchmark results were benchmarked using this patch: https://github.com/google/gvisor/compare/master...EtiennePerot:gvisor:autoresearch/recv-buffer-slots

Add value by beating these benchmark numbers, or reducing any regressions they cause, or both 
goos: linux
goarch: amd64
cpu: INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz
pkg: gvisor.dev/gvisor/test/benchmarks/base
             │                before                │                            after                            │
             │ average_container_memory_usage.bytes │ average_container_memory_usage.bytes  vs base               │
SizeEmpty-16                            14.94M ± 1%                            14.45M ± 1%  -3.30% (p=0.000 n=16)
SizeNginx-16                            23.94M ± 1%                            23.27M ± 1%  -2.80% (p=0.000 n=16)
SizeNode-16                             48.75M ± 0%                            48.28M ± 0%  -0.97% (p=0.000 n=16)
geomean                                 25.93M                                 25.32M       -2.36%

pkg: gvisor.dev/gvisor/test/benchmarks/database
                              │   before    │               after                │
                              │   sec/op    │   sec/op     vs base               │
Redis/operation.SET-16          32.36µ ± 2%   32.26µ ± 1%       ~ (p=0.867 n=16)
Redis/operation.LPUSH-16        32.54µ ± 2%   32.01µ ± 2%       ~ (p=0.283 n=16)
Redis/operation.LRANGE_100-16   50.36µ ± 1%   50.54µ ± 1%       ~ (p=0.696 n=16)
geomean                         37.57µ        37.37µ       -0.53%

                       │   before    │               after                │
                       │   SET.rps   │   SET.rps    vs base               │
Redis/operation.SET-16   67.23k ± 1%   67.57k ± 1%  +0.51% (p=0.013 n=16)

                         │   before    │             after             │
                         │  LPUSH.rps  │  LPUSH.rps   vs base          │
Redis/operation.LPUSH-16   67.45k ± 1%   67.91k ± 1%  ~ (p=0.095 n=16)

                              │     before     │              after               │
                              │ LRANGE_100.rps │ LRANGE_100.rps  vs base          │
Redis/operation.LRANGE_100-16      55.33k ± 1%      55.40k ± 1%  ~ (p=0.164 n=16)

pkg: gvisor.dev/gvisor/test/benchmarks/network
                                                                 │    before    │                after                │
                                                                 │    sec/op    │    sec/op     vs base               │
IperfOneConnection/operation.Upload-16                             2.569µ ± 38%   1.607µ ± 60%       ~ (p=0.157 n=16)
IperfOneConnection/operation.Download-16                           1.580µ ± 59%   2.502µ ± 39%       ~ (p=0.274 n=16)
IperfManyConnections/operation.Upload/parallel.4-16                2.497µ ±  1%   2.485µ ± 40%       ~ (p=0.174 n=16)
IperfManyConnections/operation.Download/parallel.4-16              2.483µ ±  1%   2.476µ ±  1%       ~ (p=0.533 n=16)
IperfManyConnections/operation.Upload/parallel.16-16               2.422µ ±  0%   2.422µ ±  0%       ~ (p=0.773 n=16)
IperfManyConnections/operation.Download/parallel.16-16             2.515µ ±  1%   2.522µ ±  1%       ~ (p=0.151 n=16)
IperfManyConnections/operation.Upload/parallel.64-16               2.430µ ±  0%   2.435µ ±  1%       ~ (p=0.831 n=16)
IperfManyConnections/operation.Download/parallel.64-16             2.622µ ± 36%   2.590µ ± 38%       ~ (p=0.468 n=16)
NginxDocSize/filesize.10Mb/concurrency.1/filesystem.tmpfs-16       7.515m ±  1%   7.504m ±  1%       ~ (p=0.239 n=16)
NginxDocSize/filesize.10Mb/concurrency.100/filesystem.tmpfs-16     7.089m ±  1%   7.018m ±  1%       ~ (p=0.086 n=16)
NginxDocSize/filesize.10Mb/concurrency.1000/filesystem.tmpfs-16    6.441m ±  2%   6.381m ±  4%       ~ (p=0.196 n=16)
NginxDocSize/filesize.1Kb/concurrency.1/filesystem.tmpfs-16        391.0µ ±  1%   390.7µ ±  3%       ~ (p=0.696 n=16)
NginxDocSize/filesize.1Kb/concurrency.100/filesystem.tmpfs-16      327.9µ ±  4%   333.9µ ±  4%       ~ (p=0.539 n=16)
NginxDocSize/filesize.1Kb/concurrency.1000/filesystem.tmpfs-16     406.6µ ±  3%   396.3µ ±  5%       ~ (p=0.184 n=16)
NginxDocSize/filesize.100Kb/concurrency.1/filesystem.tmpfs-16      497.7µ ±  2%   494.9µ ±  2%       ~ (p=0.196 n=16)
NginxDocSize/filesize.100Kb/concurrency.100/filesystem.tmpfs-16    400.7µ ±  3%   394.2µ ±  3%       ~ (p=0.254 n=16)
NginxDocSize/filesize.100Kb/concurrency.1000/filesystem.tmpfs-16   584.3µ ± 23%   474.4µ ± 21%       ~ (p=0.171 n=16)
NginxDocSize/filesize.10Mb/concurrency.1/filesystem.bind-16        8.159m ±  1%   8.145m ±  1%       ~ (p=0.381 n=16)
NginxDocSize/filesize.10Mb/concurrency.100/filesystem.bind-16      7.882m ±  1%   7.837m ±  0%       ~ (p=0.149 n=16)
NginxDocSize/filesize.10Mb/concurrency.1000/filesystem.bind-16     7.483m ±  1%   7.553m ±  3%       ~ (p=0.669 n=16)
NginxDocSize/filesize.1Kb/concurrency.1/filesystem.bind-16         394.2µ ±  1%   395.6µ ±  2%       ~ (p=0.926 n=16)
NginxDocSize/filesize.1Kb/concurrency.100/filesystem.bind-16       336.7µ ±  3%   338.4µ ±  4%       ~ (p=0.867 n=16)
NginxDocSize/filesize.1Kb/concurrency.1000/filesystem.bind-16      404.8µ ±  3%   424.2µ ±  5%       ~ (p=0.067 n=16)
NginxDocSize/filesize.100Kb/concurrency.1/filesystem.bind-16       510.7µ ±  2%   508.0µ ±  3%       ~ (p=0.642 n=16)
NginxDocSize/filesize.100Kb/concurrency.100/filesystem.bind-16     423.2µ ±  2%   411.4µ ±  3%       ~ (p=0.051 n=16)
NginxDocSize/filesize.100Kb/concurrency.1000/filesystem.bind-16    473.2µ ± 22%   483.6µ ± 19%       ~ (p=0.616 n=16)
geomean                                                            166.1µ         164.6µ        -0.92%

                                                       │           before           │                       after                        │
                                                       │ bandwidth.bytes_per_second │ bandwidth.bytes_per_second  vs base                │
IperfOneConnection/operation.Upload-16                                 3.443G ±  3%                3.640G ±   9%        ~ (p=0.275 n=16)
IperfOneConnection/operation.Download-16                              3828.5M ± 78%                834.6M ± 439%        ~ (p=0.341 n=16)
IperfManyConnections/operation.Upload/parallel.4-16                    1.446G ± 10%                1.446G ±  10%        ~ (p=0.669 n=16)
IperfManyConnections/operation.Download/parallel.4-16                  1.504G ± 13%                1.435G ±  10%        ~ (p=0.838 n=16)
IperfManyConnections/operation.Upload/parallel.16-16                   789.9M ±  1%                786.7M ±   1%        ~ (p=0.128 n=16)
IperfManyConnections/operation.Download/parallel.16-16                 492.0M ± 14%                561.2M ±  12%        ~ (p=0.110 n=16)
IperfManyConnections/operation.Upload/parallel.64-16                   188.3M ±  3%                194.8M ±   8%   +3.44% (p=0.021 n=16)
IperfManyConnections/operation.Download/parallel.64-16                 158.4M ± 20%                159.6M ±  19%        ~ (p=0.867 n=16)
geomean                                                                871.4M                      736.4M         -15.48%

                                                                 │         before          │                     after                      │
                                                                 │ requests_per_second.QPS │ requests_per_second.QPS  vs base               │
NginxDocSize/filesize.10Mb/concurrency.1/filesystem.tmpfs-16                   138.4 ±  1%               138.6 ±  0%       ~ (p=0.327 n=16)
NginxDocSize/filesize.10Mb/concurrency.100/filesystem.tmpfs-16                 147.0 ±  1%               148.8 ±  1%       ~ (p=0.081 n=16)
NginxDocSize/filesize.10Mb/concurrency.1000/filesystem.tmpfs-16                162.7 ±  1%               164.2 ±  4%       ~ (p=0.171 n=16)
NginxDocSize/filesize.1Kb/concurrency.1/filesystem.tmpfs-16                   9.057k ±  1%              9.068k ±  1%       ~ (p=0.361 n=16)
NginxDocSize/filesize.1Kb/concurrency.100/filesystem.tmpfs-16                 17.67k ± 25%              18.62k ± 16%       ~ (p=1.000 n=16)
NginxDocSize/filesize.1Kb/concurrency.1000/filesystem.tmpfs-16                8.290k ±  9%              8.525k ± 13%       ~ (p=0.287 n=16)
NginxDocSize/filesize.100Kb/concurrency.1/filesystem.tmpfs-16                 4.698k ±  1%              4.722k ±  0%       ~ (p=0.243 n=16)
NginxDocSize/filesize.100Kb/concurrency.100/filesystem.tmpfs-16               8.441k ±  4%              8.700k ±  4%       ~ (p=0.341 n=16)
NginxDocSize/filesize.100Kb/concurrency.1000/filesystem.tmpfs-16              3.330k ± 72%              5.355k ± 37%       ~ (p=0.121 n=16)
NginxDocSize/filesize.10Mb/concurrency.1/filesystem.bind-16                    126.9 ±  1%               127.2 ±  0%       ~ (p=0.345 n=16)
NginxDocSize/filesize.10Mb/concurrency.100/filesystem.bind-16                  131.8 ±  1%               132.4 ±  1%       ~ (p=0.180 n=16)
NginxDocSize/filesize.10Mb/concurrency.1000/filesystem.bind-16                 138.8 ±  2%               138.1 ±  2%       ~ (p=0.744 n=16)
NginxDocSize/filesize.1Kb/concurrency.1/filesystem.bind-16                    8.567k ±  1%              8.601k ±  1%       ~ (p=0.396 n=16)
NginxDocSize/filesize.1Kb/concurrency.100/filesystem.bind-16                  17.57k ± 20%              18.58k ± 12%       ~ (p=0.210 n=16)
NginxDocSize/filesize.1Kb/concurrency.1000/filesystem.bind-16                 7.755k ± 13%              7.064k ± 10%       ~ (p=0.239 n=16)
NginxDocSize/filesize.100Kb/concurrency.1/filesystem.bind-16                  4.356k ±  0%              4.359k ±  1%       ~ (p=0.752 n=16)
NginxDocSize/filesize.100Kb/concurrency.100/filesystem.bind-16                7.274k ±  2%              7.456k ±  5%       ~ (p=0.254 n=16)
NginxDocSize/filesize.100Kb/concurrency.1000/filesystem.bind-16               5.224k ± 38%              5.090k ± 33%       ~ (p=0.956 n=16)
geomean                                                                       1.999k                    2.065k        +3.29%

pkg: sandbox memory harness (nginx under runsc, per-container cgroup)
                                │    before    │                 after                  │
                                │    anon-B    │    anon-B     vs base                  │
SandboxMemory/phase=boot          13.07Mi ± 1%   12.72Mi ± 2%  -2.67% (p=0.000 n=42+48)
SandboxMemory/phase=rpc           15.54Mi ± 1%   15.20Mi ± 1%  -2.19% (p=0.000 n=41+48)
SandboxMemory/phase=bulk          18.38Mi ± 1%   18.39Mi ± 1%       ~ (p=0.885 n=41+48)
SandboxMemory/phase=bulkrpc       20.76Mi ± 2%   20.25Mi ± 2%  -2.43% (p=0.015 n=41+48)
SandboxMemory/phase=restored      14.76Mi ± 1%   14.41Mi ± 1%  -2.38% (p=0.000 n=48+52)
SandboxMemory/phase=restoredrpc   16.41Mi ± 1%   16.24Mi ± 1%       ~ (p=0.307 n=48+52)
geomean                           16.30Mi        16.01Mi       -1.78%

                                │    before    │                 after                  │
                                │ nonreclaim-B │ nonreclaim-B  vs base                  │
SandboxMemory/phase=boot          15.49Mi ± 1%   15.03Mi ± 1%  -2.95% (p=0.000 n=42+48)
SandboxMemory/phase=rpc           17.88Mi ± 1%   17.59Mi ± 1%  -1.58% (p=0.003 n=41+48)
SandboxMemory/phase=bulk          20.53Mi ± 1%   20.55Mi ± 0%       ~ (p=0.780 n=41+48)
SandboxMemory/phase=bulkrpc       23.18Mi ± 2%   22.73Mi ± 2%  -1.92% (p=0.039 n=41+48)
SandboxMemory/phase=restored      16.77Mi ± 1%   16.58Mi ± 1%  -1.15% (p=0.002 n=48+52)
SandboxMemory/phase=restoredrpc   18.45Mi ± 1%   18.44Mi ± 1%       ~ (p=0.254 n=48+52)
geomean                           18.55Mi        18.32Mi       -1.26%

pkg: sandbox-receive iperf3 harness (server in runsc, client on host)
                                           │   before    │               after                │
                                           │    Gbps     │    Gbps      vs base               │
IperfReceive/default/cpus=none/streams=1      35.38 ± 4%    35.47 ± 3%       ~ (p=0.353 n=10)
IperfReceive/default/cpus=none/streams=4      40.07 ± 1%    40.32 ± 1%       ~ (p=0.912 n=10)
IperfReceive/default/cpus=1/streams=1        10.343 ± 6%    9.955 ± 6%       ~ (p=0.529 n=10)
IperfReceive/default/cpus=1/streams=4         14.90 ± 2%    14.94 ± 3%       ~ (p=0.853 n=10)
IperfReceive/default/cpus=0.25/streams=1      2.659 ± 4%    2.664 ± 5%       ~ (p=0.684 n=10)
IperfReceive/default/cpus=0.25/streams=4      3.625 ± 2%    3.656 ± 2%       ~ (p=0.796 n=10)
IperfReceive/gso=off/cpus=none/streams=1      35.51 ± 1%    35.49 ± 0%       ~ (p=0.912 n=10)
IperfReceive/gso=off/cpus=none/streams=4      40.64 ± 1%    40.68 ± 1%       ~ (p=0.896 n=10)
IperfReceive/gso=off/cpus=1/streams=4         14.91 ± 1%    14.99 ± 3%       ~ (p=0.481 n=10)
IperfReceive/mtu-sized/cpus=none/streams=1    3.792 ± 2%    3.802 ± 3%       ~ (p=0.971 n=10)
IperfReceive/mtu-sized/cpus=none/streams=4    3.747 ± 2%    3.697 ± 3%       ~ (p=0.448 n=10)
IperfReceive/mtu-sized/cpus=1/streams=1       1.559 ± 4%    1.572 ± 4%       ~ (p=0.315 n=10)
IperfReceive/mtu-sized/cpus=1/streams=4       1.775 ± 4%    1.795 ± 4%       ~ (p=0.739 n=10)
IperfReceive/mtu-sized/cpus=0.25/streams=4   446.7m ± 4%   446.6m ± 3%       ~ (p=0.796 n=10)
geomean                                       7.080         7.079       -0.01%

                                           │   before    │               after                │
                                           │ cpu-sec/GiB │ cpu-sec/GiB  vs base               │
IperfReceive/default/cpus=none/streams=1     854.9m ± 5%   854.9m ± 4%       ~ (p=0.404 n=10)
IperfReceive/default/cpus=none/streams=4     842.6m ± 1%   838.2m ± 2%       ~ (p=0.853 n=10)
IperfReceive/default/cpus=1/streams=1        835.5m ± 6%   867.4m ± 6%       ~ (p=0.617 n=10)
IperfReceive/default/cpus=1/streams=4        578.2m ± 2%   577.2m ± 3%       ~ (p=0.796 n=10)
IperfReceive/default/cpus=0.25/streams=1     800.4m ± 5%   805.4m ± 4%       ~ (p=0.912 n=10)
IperfReceive/default/cpus=0.25/streams=4     587.1m ± 2%   584.9m ± 2%       ~ (p=0.971 n=10)
IperfReceive/gso=off/cpus=none/streams=1     852.1m ± 1%   853.2m ± 0%       ~ (p=0.869 n=10)
IperfReceive/gso=off/cpus=none/streams=4     831.2m ± 1%   830.7m ± 1%       ~ (p=0.739 n=10)
IperfReceive/gso=off/cpus=1/streams=4        578.5m ± 1%   575.5m ± 3%       ~ (p=0.684 n=10)
IperfReceive/mtu-sized/cpus=none/streams=1    8.311 ± 2%    8.302 ± 3%       ~ (p=0.796 n=10)
IperfReceive/mtu-sized/cpus=none/streams=4    10.46 ± 2%    10.60 ± 4%       ~ (p=0.912 n=10)
IperfReceive/mtu-sized/cpus=1/streams=1       5.535 ± 3%    5.496 ± 4%       ~ (p=0.353 n=10)
IperfReceive/mtu-sized/cpus=1/streams=4       4.883 ± 4%    4.824 ± 4%       ~ (p=0.796 n=10)
IperfReceive/mtu-sized/cpus=0.25/streams=4    4.884 ± 5%    4.886 ± 3%       ~ (p=0.684 n=10)
geomean                                       1.607         1.610       +0.16%
Benchmark environment

GCE c4-standard-16, COS cos-129-19506-448-8 (kernel 6.12.105+), hyperdisk-balanced, Docker with the systemd cgroup driver (cgroup v2), default runsc flags (systrap). before is master at 2511bc60218a, and after is the patch above. The two arms ran interleaved on the same VM.

  • The //test/benchmarks rows are the upstream binaries with runsc registered as Docker runtimes, n=16 per arm. Only the ns/op, bandwidth, QPS and rps metrics are shown. httpd, node and ruby were also run, with no p < 0.01 changes.
  • iperf OneConnection is bimodal on both arms because of an existing ~1 s close stall at the end of some runs.
  • The sandbox memory harness reads the per-container cgroup after each phase: boot is 3 s after start, rpc follows 300 sequential 1 KiB GETs from the host, bulk follows a 512 MiB download inside the sandbox, bulkrpc follows another 300 GETs, and restored/restoredrpc come after a checkpoint and restore following the download. anon-B is the cgroup's anon, and nonreclaim-B is memory.current - file - slab_reclaimable. One sample is one fresh container.
  • The receive harness runs an iperf3 server in the sandbox and a host-network client for 10 s per sample. gso=off passes --gso=false --software-gso=false to runsc, and mtu-sized disables TSO/GSO on the host bridge and veth so the sandbox receives MTU-sized frames.

Activity

1
Aaryan-Dadu6d ago

Hey @EtiennePerot, I'd like to take this one. Plan is to start with one active slot and grow it when recvmmsg fills all active slots.