Optimization: Run the systrap fast-path state machine inline instead of on a 400 µs timer

Open

The idea

systrap decides whether the Sentry and stub threads should spin waiting for each other (the "fast path") in a state machine that runs on a dedicated goroutine, waking every 400 µs while the sandbox is active and parking only when it goes idle. Under a CPU quota those timer wakeups cost host context switches and CPU in every sandbox, and on a loaded host the timer oversleeps (1.6-1.9 ms between evaluations was measured), so its decisions lag behind the workload.

systrap could instead evaluate the state machine inline from contextQueue.add, at most once per recording period: it's just an atomic timestamp check in the common case, a TryLock keeps concurrent submitters from evaluating twice. If two whole periods passed with no submission, it resets the fast path to off (as parking does today). Without that reset, a bad "on" state has no way out, and hackbench pipe/thread gets 1.8x slower. This is complementary to not spinning for contexts whose app runs outlast the spin (the bigger cost of the fast path today): that removes the spinning, this removes the timer.

Performance impact

CPU-bound sandboxes under a quota: Sentry/host overhead per unit of app CPU -10..-23% on 2-4 vCPU hosts, and -10..-13% at 0.5-2 CPU on 16 vCPUs, -4% at 0.25 CPU on 16-48 vCPUs, neutral at 1 CPU on 48. Host context switches -8..-14% on 16-48 vCPU hosts. Dense getppid() loops unchanged.
The bad: Loops that sleep 1 ms between short syscall bursts lose 2-5% throughput and get +22..78% host context switches, because the precise idle detection now resets the fast path after each sleep.
//test/benchmarks: fio libaio 4K read on rootfs -3.4% bandwidth (other libaio rows lean the same way, -1.5..-3.3%, p>0.01), node QPS -2.0% at concurrency 1 and -1.4% at concurrency 5.
Also in //test/benchmarks: sysbench Mutex -7.5%, hackbench pipe/thread -2.4%, ffmpeg -0.9%, redis max latency -8..-15%; syscallbench, startup, nginx, httpd, ruby, redis rps and iperf are neutral.

Beat the autoresearch

The following benchmark results were benchmarked using this patch: https://github.com/google/gvisor/compare/master...EtiennePerot:gvisor:autoresearch/inline-fastpath-monitor

Add value by beating these benchmark numbers, or reducing any regressions they cause, or both

goos: linux
goarch: amd64
pkg: gvisor.dev/gvisor/test/benchmarks/base
                                                      │   before    │                after                │
                                                      │   sec/op    │    sec/op     vs base               │
StartupEmpty-16                                         182.1m ± 2%   180.8m ±  2%       ~ (p=0.210 n=16)
StartupNginx-16                                         499.7m ± 1%   501.5m ±  3%       ~ (p=0.669 n=16)
SizeEmpty-16                                            542.3m ± 2%   542.3m ±  2%       ~ (p=0.838 n=16)
Sysbench/testname.CPU-16                                520.2µ ± 1%   516.8µ ±  2%       ~ (p=0.445 n=16)
Sysbench/testname.CPU/threads.16-16                     65.85µ ± 1%   66.48µ ±  2%       ~ (p=0.138 n=16)
Sysbench/testname.Memory-16                              1.355 ± 0%    1.349 ±  0%       ~ (p=0.128 n=16)
Sysbench/testname.Memory/threads.16-16                   1.284 ± 1%    1.287 ±  1%       ~ (p=0.210 n=16)
Sysbench/testname.Mutex-16                              5.340µ ± 4%   4.939µ ±  3%  -7.51% (p=0.001 n=16)
Syscallbench/syscall.getpid-16                          3.464µ ± 1%   3.473µ ±  2%       ~ (p=0.439 n=16)
Syscallbench/syscall.getpidopt-16                       1.175µ ± 3%   1.188µ ±  2%       ~ (p=0.224 n=16)
Syscallbench/syscall.getpid/run=2-16                    2.959µ ± 2%   2.950µ ±  2%       ~ (p=1.000 n=16)
Syscallbench/syscall.getpidopt/run=2-16                 1.028µ ± 3%   1.044µ ±  4%       ~ (p=0.759 n=16)
Hackbench/ipcMode.pipe/processMode.thread-16            5.468m ± 1%   5.336m ±  2%  -2.41% (p=0.001 n=16)
Hackbench/ipcMode.socket/processMode.process-16         12.77m ± 7%   13.42m ± 10%       ~ (p=0.270 n=16)
Hackbench/ipcMode.pipe/processMode.thread/run=2-16      4.770m ± 2%   4.673m ±  3%  -2.04% (p=0.029 n=16)
Hackbench/ipcMode.socket/processMode.process/run=2-16   9.868m ± 5%   9.798m ± 11%       ~ (p=0.381 n=16)
geomean                                                 1.497m        1.492m        -0.34%

pkg: gvisor.dev/gvisor/test/benchmarks/database
                              │     before     │                 after                  │
                              │ max_latency_ms │ max_latency_ms  vs base                │
Redis/operation.LPUSH-16           3.355 ± 14%      2.867 ± 11%  -14.55% (p=0.001 n=16)
Redis/operation.LRANGE_100-16      2.967 ± 19%      2.883 ± 12%        ~ (p=0.649 n=16)
Redis/operation.SET-16             3.063 ± 15%      2.823 ±  8%   -7.84% (p=0.008 n=16)
geomean                            3.124            2.858         -8.53%

                              │   before    │               after                │
                              │     rps     │     rps      vs base               │
Redis/operation.LPUSH-16        58.79k ± 2%   58.62k ± 1%       ~ (p=0.696 n=16)
Redis/operation.LRANGE_100-16   46.70k ± 1%   46.63k ± 1%       ~ (p=0.583 n=16)
Redis/operation.SET-16          58.92k ± 1%   58.71k ± 1%       ~ (p=0.361 n=16)
geomean                         54.49k        54.34k       -0.27%

pkg: gvisor.dev/gvisor/test/benchmarks/fs
                                                                                                │           before           │                       after                       │
                                                                                                │ bandwidth.bytes_per_second │ bandwidth.bytes_per_second  vs base               │
FioRead/operation.read/ioEngine.libaio/jobs.1/blockSize.4K/directIO.false/filesystem.rootfs-16                   353.5M ± 1%                  341.6M ± 1%  -3.39% (p=0.000 n=16)
FioRead/operation.read/ioEngine.libaio/jobs.1/blockSize.4K/directIO.false/filesystem.tmpfs-16                    369.9M ± 2%                  364.5M ± 1%  -1.45% (p=0.032 n=16)
FioRead/operation.read/ioEngine.libaio/jobs.8/blockSize.4K/directIO.true/filesystem.rootfs-16                    76.46M ± 2%                  75.39M ± 2%       ~ (p=0.184 n=16)
FioRead/operation.read/ioEngine.libaio/jobs.8/blockSize.4K/directIO.true/filesystem.tmpfs-16                     78.64M ± 3%                  76.04M ± 3%  -3.31% (p=0.035 n=16)
FioWrite/operation.write/ioEngine.libaio/jobs.8/blockSize.4K/directIO.true/filesystem.rootfs-16                  77.44M ± 4%                  74.89M ± 6%       ~ (p=0.100 n=16)
FioWrite/operation.write/ioEngine.libaio/jobs.8/blockSize.4K/directIO.true/filesystem.tmpfs-16                   78.55M ± 5%                  76.22M ± 2%       ~ (p=0.224 n=16)
FioWrite/operation.write/ioEngine.sync/jobs.1/blockSize.4K/directIO.false/filesystem.rootfs-16                   999.6M ± 1%                 1000.3M ± 1%       ~ (p=0.956 n=16)
FioWrite/operation.write/ioEngine.sync/jobs.1/blockSize.4K/directIO.false/filesystem.tmpfs-16                    1.628G ± 1%                  1.611G ± 2%       ~ (p=0.287 n=16)
geomean                                                                                                          229.8M                       225.0M       -2.10%

pkg: gvisor.dev/gvisor/test/benchmarks/media
          │   before   │              after               │
          │   sec/op   │   sec/op    vs base              │
Ffmpeg-16   28.82 ± 0%   28.57 ± 0%  -0.87% (p=0.002 n=8)

pkg: gvisor.dev/gvisor/test/benchmarks/network
                                                    │           before           │                       after                       │
                                                    │ bandwidth.bytes_per_second │ bandwidth.bytes_per_second  vs base               │
IperfOneConnection/operation.Download/bytes=10GB-16                  4.377G ± 2%                  4.407G ± 2%       ~ (p=0.590 n=16)
IperfOneConnection/operation.Upload/bytes=10GB-16                    3.931G ± 2%                  3.894G ± 2%       ~ (p=0.539 n=16)
geomean                                                              4.148G                       4.143G       -0.14%

                                                               │         before          │                     after                      │
                                                               │ requests_per_second.QPS │ requests_per_second.QPS  vs base               │
NginxDocSize/filesize.10Kb/concurrency.1/filesystem.bind-16                  7.068k ± 1%               7.100k ± 1%       ~ (p=0.171 n=16)
NginxDocSize/filesize.10Kb/concurrency.1/filesystem.tmpfs-16                 7.620k ± 0%               7.649k ± 1%       ~ (p=0.116 n=16)
NginxDocSize/filesize.10Kb/concurrency.100/filesystem.bind-16                21.55k ± 1%               21.28k ± 2%       ~ (p=0.539 n=16)
NginxDocSize/filesize.10Kb/concurrency.100/filesystem.tmpfs-16               26.86k ± 1%               26.53k ± 2%       ~ (p=0.539 n=16)
NginxDocSize/filesize.1Mb/concurrency.1/filesystem.bind-16                    975.4 ± 1%                978.2 ± 1%       ~ (p=0.724 n=16)
NginxDocSize/filesize.1Mb/concurrency.1/filesystem.tmpfs-16                  1.087k ± 2%               1.077k ± 1%       ~ (p=0.125 n=16)
NginxDocSize/filesize.1Mb/concurrency.100/filesystem.bind-16                 1.091k ± 2%               1.092k ± 2%       ~ (p=0.291 n=16)
NginxDocSize/filesize.1Mb/concurrency.100/filesystem.tmpfs-16                1.246k ± 1%               1.241k ± 2%       ~ (p=0.235 n=16)
Httpd/filesize.10Kb/concurrency.1-16                                         2.733k ± 2%               2.779k ± 2%  +1.68% (p=0.016 n=16)
Httpd/filesize.10Kb/concurrency.100-16                                        19.89 ± 0%                19.87 ± 0%       ~ (p=0.115 n=16)
Httpd/filesize.1Mb/concurrency.1-16                                           201.3 ± 1%                202.1 ± 1%       ~ (p=0.717 n=16)
Httpd/filesize.1Mb/concurrency.100-16                                         19.77 ± 0%                19.76 ± 0%       ~ (p=0.687 n=16)
Node/concurrency.1-16                                                         910.1 ± 2%                891.8 ± 2%  -2.01% (p=0.000 n=16)
Node/concurrency.10-16                                                       1.295k ± 2%               1.302k ± 3%       ~ (p=0.860 n=16)
Node/concurrency.25-16                                                       1.488k ± 3%               1.435k ± 4%  -3.53% (p=0.024 n=16)
Node/concurrency.5-16                                                        1.265k ± 2%               1.248k ± 2%  -1.38% (p=0.003 n=16)
Ruby/concurrency.1-16                                                        1.085k ± 2%               1.082k ± 5%       ~ (p=0.846 n=16)
Ruby/concurrency.10-16                                                       3.693k ± 1%               3.669k ± 0%       ~ (p=0.206 n=16)
Ruby/concurrency.25-16                                                       3.982k ± 1%               3.970k ± 1%       ~ (p=0.521 n=16)
Ruby/concurrency.5-16                                                        3.124k ± 1%               3.128k ± 0%       ~ (p=0.948 n=16)
geomean                                                                      1.418k                    1.412k       -0.41%

pkg: many-sandbox steady-state harness (perf over 15 s window)
                                        │      before      │                  after                   │
                                        │ overhead/app-cpu │ overhead/app-cpu  vs base                │
Burner/vcpus=2/sandboxes=5/cpu=0.25          121.20m ±  6%       93.16m ± 10%  -23.13% (p=0.002 n=6)
Burner/vcpus=2/sandboxes=5/cpu=1             107.52m ±  7%       97.05m ± 18%   -9.73% (p=0.041 n=6)
Tick1ms/vcpus=2/sandboxes=1/cpu=0.25           8.403 ± 53%        6.594 ± 57%        ~ (p=0.394 n=6)
Tick1ms/vcpus=2/sandboxes=1/cpu=1              9.159 ± 56%        5.728 ± 45%        ~ (p=0.132 n=6)
Tick10ms/vcpus=2/sandboxes=7/cpu=1             6.596 ±  3%        6.359 ±  2%   -3.60% (p=0.015 n=6)
Burner/vcpus=4/sandboxes=10/cpu=0.25          130.9m ±  4%       107.2m ±  6%  -18.13% (p=0.002 n=6)
Burner/vcpus=4/sandboxes=10/cpu=1            123.00m ±  7%       99.59m ±  9%  -19.04% (p=0.002 n=6)
Tick1ms/vcpus=4/sandboxes=2/cpu=0.25           6.793 ± 19%        6.907 ± 19%        ~ (p=0.699 n=6)
Tick1ms/vcpus=4/sandboxes=2/cpu=1              6.496 ± 24%        6.933 ± 16%        ~ (p=0.485 n=6)
Tick10ms/vcpus=4/sandboxes=14/cpu=1            6.501 ±  2%        6.461 ±  2%        ~ (p=0.394 n=6)
Burner/vcpus=16/sandboxes=28/cpu=0.5          150.0m ±  8%       129.9m ± 15%  -13.40% (p=0.026 n=6)
Burner/vcpus=16/sandboxes=40/cpu=0.25         168.5m ±  2%       162.5m ±  2%   -3.57% (p=0.000 n=12)
Burner/vcpus=16/sandboxes=40/cpu=1            133.1m ±  9%       120.3m ±  3%   -9.65% (p=0.000 n=12)
Burner/vcpus=16/sandboxes=40/cpu=2            135.7m ± 22%       118.4m ±  4%  -12.72% (p=0.002 n=6)
Tick1ms/vcpus=16/sandboxes=8/cpu=0.25          6.310 ±  1%        6.352 ±  5%        ~ (p=0.699 n=6)
Tick1ms/vcpus=16/sandboxes=8/cpu=1             6.550 ±  3%        6.604 ±  2%        ~ (p=0.378 n=12)
Tick10ms/vcpus=16/sandboxes=56/cpu=1           5.735 ±  1%        5.558 ±  2%   -3.10% (p=0.002 n=6)
Sparse50ms/vcpus=16/sandboxes=56/cpu=1         7.177 ±  2%        7.150 ±  4%        ~ (p=0.485 n=6)
Burner/vcpus=48/sandboxes=120/cpu=0.25        176.7m ±  1%       169.8m ±  2%   -3.90% (p=0.002 n=6)
Burner/vcpus=48/sandboxes=120/cpu=1           151.6m ±  6%       154.8m ±  4%        ~ (p=0.485 n=6)
Sparse50ms/vcpus=48/sandboxes=120/cpu=1        7.006 ±  1%        7.193 ±  1%   +2.67% (p=0.004 n=6)
Tick1ms/vcpus=48/sandboxes=24/cpu=1            6.682 ±  1%        6.784 ±  1%   +1.52% (p=0.009 n=6)
geomean                                        1.167              1.072         -8.13%

                                        │    before    │                 after                 │
                                        │ host-ctxsw/s │ host-ctxsw/s   vs base                │
Burner/vcpus=2/sandboxes=5/cpu=0.25       12.86k ±  6%    12.71k ±  2%        ~ (p=0.937 n=6)
Burner/vcpus=2/sandboxes=5/cpu=1          13.66k ±  3%    12.06k ± 17%  -11.70% (p=0.041 n=6)
Getppid/vcpus=2/sandboxes=1/cpu=1         7.017k ± 23%    9.897k ± 23%  +41.04% (p=0.009 n=6)
Tick1ms/vcpus=2/sandboxes=1/cpu=0.25      4.567k ±  8%    7.213k ±  3%  +57.91% (p=0.002 n=6)
Tick1ms/vcpus=2/sandboxes=1/cpu=1         11.97k ±  6%    21.36k ±  7%  +78.39% (p=0.002 n=6)
Tick10ms/vcpus=2/sandboxes=7/cpu=1        410.7k ±  2%    421.4k ±  2%   +2.61% (p=0.009 n=6)
Burner/vcpus=4/sandboxes=10/cpu=0.25      25.73k ±  3%    25.45k ±  2%        ~ (p=0.394 n=6)
Burner/vcpus=4/sandboxes=10/cpu=1         25.78k ±  5%    25.51k ±  7%        ~ (p=0.589 n=6)
Getppid/vcpus=4/sandboxes=1/cpu=1         7.580k ± 11%    4.501k ±  8%  -40.62% (p=0.002 n=6)
Tick1ms/vcpus=4/sandboxes=2/cpu=0.25      9.421k ± 12%   13.905k ±  4%  +47.60% (p=0.002 n=6)
Tick1ms/vcpus=4/sandboxes=2/cpu=1         19.25k ± 12%    30.77k ± 12%  +59.83% (p=0.002 n=6)
Tick10ms/vcpus=4/sandboxes=14/cpu=1       682.4k ±  1%    687.7k ±  2%        ~ (p=0.310 n=6)
Burner/vcpus=16/sandboxes=28/cpu=0.5      98.97k ±  7%    84.86k ±  8%  -14.26% (p=0.002 n=6)
Burner/vcpus=16/sandboxes=40/cpu=0.25     132.0k ±  1%    121.4k ±  2%   -8.04% (p=0.000 n=12)
Burner/vcpus=16/sandboxes=40/cpu=1        122.0k ±  2%    109.1k ±  4%  -10.51% (p=0.000 n=12)
Burner/vcpus=16/sandboxes=40/cpu=2        126.8k ± 11%    112.3k ±  6%  -11.42% (p=0.015 n=6)
Tick1ms/vcpus=16/sandboxes=8/cpu=0.25     41.76k ± 12%    50.92k ±  5%  +21.94% (p=0.002 n=6)
Tick1ms/vcpus=16/sandboxes=8/cpu=1        98.87k ±  3%   136.19k ±  1%  +37.74% (p=0.000 n=12)
Tick10ms/vcpus=16/sandboxes=56/cpu=1      3.973M ±  0%    4.127M ±  0%   +3.86% (p=0.002 n=6)
Sparse50ms/vcpus=16/sandboxes=56/cpu=1    272.4k ±  1%    307.1k ±  3%  +12.73% (p=0.002 n=6)
Getppid/vcpus=16/sandboxes=1/cpu=1        6.100k ± 15%    3.600k ±  8%  -40.98% (p=0.002 n=6)
Burner/vcpus=48/sandboxes=120/cpu=0.25    400.5k ±  0%    359.1k ±  0%  -10.34% (p=0.002 n=6)
Burner/vcpus=48/sandboxes=120/cpu=1       377.2k ±  1%    335.4k ±  3%  -11.08% (p=0.002 n=6)
Sparse50ms/vcpus=48/sandboxes=120/cpu=1   636.1k ±  0%    512.2k ±  1%  -19.48% (p=0.002 n=6)
Tick1ms/vcpus=48/sandboxes=24/cpu=1       312.1k ±  1%    401.6k ±  2%  +28.69% (p=0.002 n=6)
geomean                                   65.79k          68.69k         +4.42%

                                        │   before    │                after                │
                                        │   work/s    │   work/s     vs base                │
Getppid/vcpus=2/sandboxes=1/cpu=1         152.8k ± 1%   151.5k ± 1%        ~ (p=0.065 n=6)
Tick1ms/vcpus=2/sandboxes=1/cpu=0.25      24.67k ± 2%   23.97k ± 1%   -2.83% (p=0.002 n=6)
Tick1ms/vcpus=2/sandboxes=1/cpu=1         109.0k ± 3%   107.6k ± 2%        ~ (p=0.485 n=6)
Tick10ms/vcpus=2/sandboxes=7/cpu=1        196.1k ± 2%   204.9k ± 2%   +4.48% (p=0.002 n=6)
Getppid/vcpus=4/sandboxes=1/cpu=1         138.0k ± 2%   138.5k ± 2%        ~ (p=0.699 n=6)
Tick1ms/vcpus=4/sandboxes=2/cpu=0.25      41.91k ± 6%   39.60k ± 7%   -5.51% (p=0.026 n=6)
Tick1ms/vcpus=4/sandboxes=2/cpu=1         168.9k ± 5%   168.1k ± 3%        ~ (p=0.394 n=6)
Tick10ms/vcpus=4/sandboxes=14/cpu=1       322.7k ± 1%   335.1k ± 1%   +3.82% (p=0.002 n=6)
Tick1ms/vcpus=16/sandboxes=8/cpu=0.25     230.5k ± 3%   219.7k ± 1%   -4.70% (p=0.002 n=6)
Tick1ms/vcpus=16/sandboxes=8/cpu=1        908.2k ± 2%   886.7k ± 2%   -2.37% (p=0.028 n=12)
Tick10ms/vcpus=16/sandboxes=56/cpu=1      1.912M ± 0%   2.025M ± 0%   +5.93% (p=0.002 n=6)
Sparse50ms/vcpus=16/sandboxes=56/cpu=1    1.096M ± 1%   1.226M ± 1%  +11.85% (p=0.002 n=6)
Getppid/vcpus=16/sandboxes=1/cpu=1        187.6k ± 4%   183.0k ± 7%        ~ (p=0.589 n=6)
Sparse50ms/vcpus=48/sandboxes=120/cpu=1   2.422M ± 0%   3.412M ± 1%  +40.87% (p=0.002 n=6)
Tick1ms/vcpus=48/sandboxes=24/cpu=1       2.598M ± 0%   2.523M ± 0%   -2.87% (p=0.002 n=6)
geomean                                   298.3k        305.5k        +2.43%

Benchmark environment

GCE c4-standard-2/4/16/48 (vcpus=), COS cos-129-19506-448-8 (kernel 6.12.105+), hyperdisk-balanced. before is master at 2511bc60218a, after is the patch above (some runs used a build whose runsc is identical and only the test file differs). The //test/benchmarks blocks are the real targets run on c4-standard-16 as master and patched docker runtimes (systrap, no CPU limit), interleaved; nginx/httpd and fio ran a subset of configurations, iperf was run with a 10 GB transfer, and run=2 rows are a second independent run. The steady-state harness starts N runsc run --cpu-num-from-quota sandboxes with the given cpu.max quota, runs them 45 s, and measures the last 15 s with perf record -a; overhead/app-cpu is non-app CPU divided by the workload binary's CPU, host-ctxsw/s is host context switches. Workloads: Burner burns CPU 50 ms out of every 200 ms (its own work counter is clock-budgeted and confounded by thread CPU accounting, so only overhead is shown), Tick1ms does about 20 µs of work every 1 ms, Tick10ms every 10 ms, Sparse50ms about 100 µs every 50 ms, Getppid is a dense getppid() loop. The vcpus=16 rows with n=12 pool two sessions on different VMs.

Activity

No activity yet.