The idea
pkg/state is slow and full of reflection. We should make it faster and less allocation-heavy.
The below is a vibe-coded code-generation-based approach. Maybe there's better approaches out there.
Performance impact of the below prototype (beat it or else)
runsc checkpointof a small HTTP server: -50% to -71% at every CPU quota; restore RPC -38% uncapped and -5% to -11% at 1 and 0.25 CPU; sandbox CPU per checkpoint+restore cycle -26% to -33%.- The kernel's
state.Save/state.Loadinside checkpoint and restore: -79% and -72% time, -97% and -91% heap allocations, -78% and -69% bytes allocated, and no garbage collections during either (2 and 3 before). //test/benchmarks/base:CheckpointEmpty-20%,CheckpointNginx-14%,RestoreEmpty-7%,RestoreNginx-6%;startup_teststartup andsize_testneutral;usage_testmemory -3% for the empty container (from an unused IPv4 state removal included in the series), neutral for nginx.//test/benchmarks/checkpointfile(which checkpoints nginx with 4 workers once it is idle, then decodes and re-encodes the kernel from the state file in-process): decode -64% time, -71% CPU, -87% allocations and -57% bytes allocated; re-encode -75% time, -83% CPU, -92% allocations and -64% bytes allocated; state file -37% (356 KiB to 225 KiB).- The kernel's state file (
checkpoint.img) is about 40% smaller (232 KB to 137 KB for an idle container); a 20000-node random object graph encodes to 15% fewer bytes. state.Save/state.Loadmicro-benchmarks: -39% to -86% time and -78% to -97% allocations on struct-heavy graphs, -59%/-66% time and -94%/-88% allocations on a 20000-node random graph. Saving a root-level 1024-element slice or map (still handled by the reflective path) is about 10% slower, and small graphs allocate 5-9% more bytes (slab rounding).- Costs:
runscbinary +4% (0.8 MB, mostly generated code), no measurable build-time change.
Beat the autoresearch
The following benchmark results were benchmarked using this patch: https://github.com/google/gvisor/compare/master...EtiennePerot:gvisor:autoresearch/state-codegen-v2-ec89
Add value by beating these benchmark numbers, or reducing any regressions they cause, or both
goos: linux
goarch: amd64
cpu: INTEL(R) XEON(R) PLATINUM 8581C CPU @ 2.30GHz
pkg: gvisor.dev/gvisor/pkg/state/tests
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
Encoding/fields/state 2323.3µ ± 1% 511.9µ ± 2% -77.97% (p=0.000 n=16)
Encoding/ptr/state 806.0µ ± 1% 494.3µ ± 1% -38.68% (p=0.000 n=16)
Encoding/map/state 139.3µ ± 1% 152.9µ ± 2% +9.76% (p=0.000 n=16)
Encoding/slice/state 82.50µ ± 1% 90.96µ ± 1% +10.27% (p=0.000 n=16)
Decoding/ptr/state 529.7µ ± 1% 228.1µ ± 2% -56.94% (p=0.000 n=16)
Decoding/map/state 266.0µ ± 1% 259.6µ ± 1% -2.42% (p=0.000 n=16)
Decoding/slice/state 125.8µ ± 1% 124.2µ ± 1% ~ (p=0.110 n=16)
Decoding/fields/state 2738.8µ ± 0% 393.9µ ± 1% -85.62% (p=0.000 n=16)
GraphSave 430.5m ± 1% 178.0m ± 2% -58.65% (p=0.000 n=12)
GraphLoad 267.89m ± 1% 90.48m ± 5% -66.22% (p=0.000 n=12)
geomean 1.615m 834.7µ -48.31%
│ before.txt │ after.txt │
│ B/op │ B/op vs base │
Encoding/fields/state 1844.8Ki ± 0% 654.1Ki ± 0% -64.54% (p=0.000 n=16)
Encoding/ptr/state 494.7Ki ± 0% 520.6Ki ± 0% +5.24% (p=0.000 n=16)
Encoding/map/state 121.6Ki ± 0% 128.7Ki ± 0% +5.85% (p=0.000 n=16)
Encoding/slice/state 97.56Ki ± 0% 104.66Ki ± 0% +7.28% (p=0.000 n=16)
Decoding/ptr/state 291.9Ki ± 0% 254.9Ki ± 0% -12.68% (p=0.000 n=16)
Decoding/map/state 222.2Ki ± 0% 235.0Ki ± 0% +5.73% (p=0.000 n=16)
Decoding/slice/state 134.8Ki ± 0% 147.5Ki ± 0% +9.45% (p=0.000 n=16)
Decoding/fields/state 1741.5Ki ± 0% 450.2Ki ± 0% -74.15% (p=0.000 n=16)
GraphSave 115.88Mi ± 0% 77.92Mi ± 0% -32.76% (p=0.000 n=12)
GraphLoad 100.58Mi ± 0% 45.84Mi ± 0% -54.42% (p=0.000 n=12)
geomean 1.047Mi 764.6Ki -28.71%
│ before.txt │ after.txt │
│ allocs/op │ allocs/op vs base │
Encoding/fields/state 12.976k ± 0% 2.209k ± 0% -82.98% (p=0.000 n=16)
Encoding/ptr/state 5040.0 ± 0% 161.0 ± 0% -96.81% (p=0.000 n=16)
Encoding/map/state 1.833k ± 0% 1.827k ± 0% -0.33% (p=0.000 n=16)
Encoding/slice/state 1.064k ± 0% 1.057k ± 0% -0.66% (p=0.000 n=16)
Decoding/ptr/state 5.918k ± 0% 1.036k ± 0% -82.49% (p=0.000 n=16)
Decoding/map/state 1.853k ± 0% 1.833k ± 0% -1.08% (p=0.000 n=16)
Decoding/slice/state 1.061k ± 0% 1.041k ± 0% -1.89% (p=0.000 n=16)
Decoding/fields/state 13.863k ± 0% 3.089k ± 0% -77.72% (p=0.000 n=16)
GraphSave 2013.0k ± 0% 114.3k ± 0% -94.32% (p=0.000 n=12)
GraphLoad 2221.2k ± 0% 263.1k ± 0% -88.16% (p=0.000 n=12)
geomean 12.48k 3.236k -74.08%
│ before.txt │ after.txt │
│ stream-bytes │ stream-bytes vs base │
GraphSave 6.609Mi ± 0% 5.597Mi ± 0% -15.31% (p=0.000 n=12)
pkg: gvisor.dev/gvisor/test/benchmarks/base
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
CheckpointEmpty-16 56.64m ± 5% 45.24m ± 6% -20.12% (p=0.002 n=6)
RestoreEmpty-16 178.0m ± 3% 165.1m ± 5% -7.23% (p=0.009 n=6)
CheckpointNginx-16 92.07m ± 6% 78.84m ± 4% -14.38% (p=0.002 n=6)
RestoreNginx-16 224.4m ± 3% 211.7m ± 3% -5.66% (p=0.002 n=6)
StartupEmpty-16 176.1m ± 3% 173.5m ± 1% ~ (p=0.270 n=16)
StartupNginx-16 475.3m ± 1% 473.0m ± 1% ~ (p=0.590 n=16)
SizeEmpty-16 361.5m ± 2% 364.9m ± 1% ~ (p=0.171 n=16)
SizeNginx-16 732.8m ± 1% 737.2m ± 1% ~ (p=0.184 n=16)
geomean 215.3m 201.8m -6.26%
│ before.txt │ after.txt │
│ average_container_size.bytes │ average_container_size.bytes vs base │
SizeEmpty-16 22.45M ± 13% 18.94M ± 9% ~ (p=0.051 n=16)
SizeNginx-16 31.54M ± 9% 31.46M ± 8% ~ (p=0.867 n=16)
geomean 26.61M 24.41M -8.27%
│ before.txt │ after.txt │
│ average_container_memory_usage.bytes │ average_container_memory_usage.bytes vs base │
UsageEmpty-16 14.85M ± 1% 14.45M ± 2% -2.69% (p=0.000 n=16)
UsageNginx-16 23.96M ± 0% 23.94M ± 1% ~ (p=0.381 n=16)
geomean 18.86M 18.60M -1.38%
pkg: runsc checkpoint/restore cycle harness (c4-standard-16)
│ before.txt │ after.txt │
│ sec/op │ sec/op vs base │
Checkpoint/cpu=uncapped 21.828m ± 1% 9.263m ± 1% -57.56% (n=126)
SuspendTotal/cpu=uncapped 40.81m ± 0% 28.29m ± 1% -30.67% (n=126)
Restore/cpu=uncapped 26.57m ± 1% 16.43m ± 1% -38.16% (n=120)
SandboxCreate/cpu=uncapped 16.68m ± 0% 14.93m ± 0% -10.51% (n=120)
ResumeTotal/cpu=uncapped 55.09m ± 0% 43.68m ± 1% -20.72% (n=120)
Checkpoint/cpu=1/hold 22.05m ± 2% 10.95m ± 1% -50.36% (n=126)
SuspendTotal/cpu=1/hold 42.62m ± 1% 31.55m ± 1% -25.97% (n=126)
Restore/cpu=1/hold 37.07m ± 11% 35.07m ± 0% -5.40% (p=0.000 n=120)
SandboxCreate/cpu=1/hold 18.72m ± 1% 17.12m ± 1% -8.57% (n=120)
ResumeTotal/cpu=1/hold 70.44m ± 5% 64.29m ± 0% -8.73% (n=120)
Checkpoint/cpu=0.25/hold 38.16m ± 37% 11.04m ± 2% -71.06% (n=126)
SuspendTotal/cpu=0.25/hold 86.48m ± 19% 31.76m ± 1% -63.27% (n=126)
Restore/cpu=0.25/hold 115.9m ± 0% 103.4m ± 6% -10.79% (n=120)
SandboxCreate/cpu=0.25/hold 63.06m ± 17% 17.60m ± 4% -72.09% (p=0.000 n=120)
ResumeTotal/cpu=0.25/hold 195.7m ± 5% 170.2m ± 7% -13.06% (p=0.000 n=120)
Checkpoint/cpu=1 19.130m ± 1% 9.262m ± 1% -51.58% (n=126)
SuspendTotal/cpu=1 39.06m ± 1% 28.88m ± 0% -26.06% (n=126)
Restore/cpu=1 37.29m ± 11% 35.06m ± 2% -5.98% (n=120)
SandboxCreate/cpu=1 17.53m ± 1% 15.91m ± 0% -9.25% (n=120)
ResumeTotal/cpu=1 66.80m ± 5% 62.10m ± 1% -7.03% (n=120)
geomean 40.68m 26.84m -34.02%
│ before.txt │ after.txt │
│ cpu-sec/op │ cpu-sec/op vs base │
CPUPerCycle/cpu=uncapped 118.64m ± 1% 80.06m ± 1% -32.52% (p=0.002 n=6)
CPUPerCycle/cpu=1/hold 100.17m ± 3% 72.28m ± 3% -27.85% (p=0.002 n=6)
CPUPerCycle/cpu=0.25/hold 99.20m ± 5% 72.94m ± 3% -26.47% (p=0.002 n=6)
CPUPerCycle/cpu=1 91.46m ± 2% 64.55m ± 1% -29.43% (p=0.002 n=6)
geomean 101.9m 72.25m -29.10%
pkg: gvisor.dev/gvisor/test/benchmarks/checkpointfile
│ before │ after │
│ sec/op │ sec/op vs base │
CheckpointFile/Decode-16 12.338m ± 6% 4.425m ± 7% -64.13% (p=0.000 n=16)
CheckpointFile/Encode-16 13.791m ± 4% 3.465m ± 4% -74.87% (p=0.000 n=16)
geomean 13.04m 3.916m -69.98%
│ before │ after │
│ checkpoint-total-B │ checkpoint-total-B vs base │
CheckpointFile/Decode-16 5.367Mi ± 0% 5.239Mi ± 0% -2.38% (p=0.000 n=16)
│ before │ after │
│ cpu-sec │ cpu-sec vs base │
CheckpointFile/Decode-16 11.341m ± 3% 3.329m ± 5% -70.65% (p=0.000 n=16)
CheckpointFile/Encode-16 20.449m ± 3% 3.417m ± 3% -83.29% (p=0.000 n=16)
geomean 15.23m 3.373m -77.85%
│ before │ after │
│ pages-file-B │ pages-file-B vs base │
CheckpointFile/Decode-16 5.016Mi ± 0% 5.016Mi ± 0% ~ (p=0.350 n=16)
│ before │ after │
│ pages-metadata-B │ pages-metadata-B vs base │
CheckpointFile/Decode-16 3.470Ki ± 4% 3.407Ki ± 7% ~ (p=0.075 n=16)
│ before │ after │
│ state-file-B │ state-file-B vs base │
CheckpointFile/Decode-16 356.4Ki ± 0% 225.4Ki ± 0% -36.74% (p=0.000 n=16)
│ before │ after │
│ B/op │ B/op vs base │
CheckpointFile/Decode-16 5.535Mi ± 0% 2.372Mi ± 0% -57.14% (p=0.000 n=16)
CheckpointFile/Encode-16 6.518Mi ± 0% 2.323Mi ± 0% -64.36% (p=0.000 n=16)
geomean 6.007Mi 2.348Mi -60.91%
│ before │ after │
│ allocs/op │ allocs/op vs base │
CheckpointFile/Decode-16 99.36k ± 0% 12.95k ± 0% -86.97% (p=0.000 n=16)
CheckpointFile/Encode-16 84.246k ± 0% 6.507k ± 0% -92.28% (p=0.000 n=16)
geomean 91.49k 9.178k -89.97%
│ before │ after │
│ encoded-kernel-B │ encoded-kernel-B vs base │
CheckpointFile/Encode-16 342.3Ki ± 0% 210.9Ki ± 0% -38.38% (p=0.000 n=16)Benchmark environment
GCE c4-standard-16 (Intel Xeon Platinum 8581C), COS 129 (cos-129-19506-448-8, kernel 6.12.105+, cgroup v2), hyperdisk-balanced boot disk; before = master at 2511bc60218a, after = the patch above (release builds of runsc). test/benchmarks/base: startup_test (checkpoint/restore and startup), size_test and usage_test run against both builds as Docker runtimes, interleaved, until p<0.01 or n=16 per arm. Cycle harness: a pause container plus a small HTTP server sandbox repeatedly checkpointed (runsc checkpoint) and restored (runsc restore into a new sandbox), interleaved ABBA over 6 rounds; cpu= is the sandbox's CPU quota (1/hold and 0.25/hold wait 1-2 s between cycles so that CFS throttling periods don't carry over), and CPUPerCycle is the sandbox and gofer cgroup CPU per checkpoint+restore cycle (one sample per round). Each arm restores from its own initial snapshot, so the image size (and checkpoint time) of an arm can differ by a few percent between runs; cycle samples are per checkpoint/restore within runs, so they are not fully independent. Kernel state.Save/state.Load: builds instrumented to log runtime.MemStats differences around the kernel's state.Save and state.Load calls, with debug logging for the first 21 checkpoints and restores of 6 interleaved rounds per arm (per-type stats collection is disabled in the after build, and always on in master, as shipped). test/benchmarks/checkpointfile: the benchmark from commit ec8963ba16 ("Add pkg/state benchmark for kernel serialization"), run on master at c5ab48fa9b plus that commit (before) and the same tree with the patch's commits applied on top (after); 5 decode/encode iterations per run, interleaved over 16 runs per arm, each run with a fresh nginx container and checkpoint. The other blocks were measured with the same patch on master at 2511bc60218a. pkg/state/tests: the package's Save/Load benchmarks interleaved over 16 runs pinned to one CPU, and a benchmark (not part of the patch) saving and loading a 20000-node random graph built from the types of the package's random-graph test (pointers, interior pointers, slices, maps, interfaces, fields saved through values), interleaved over 12 runs on a c4-standard-8. The patch includes two small independent changes from the same series (cheaper seccheck point registration at boot, and removal of unused IPv4 ID-generation state), which account for most of the SandboxCreate and usage_test memory differences.
Activity
3
tamird6d ago@parth-opensrc, please take the lead on the rewrite. We won’t pursue it as our primary workstream, but can help with review and focused refinements.
We’ve been testing the prototype. Two results look useful for the next revision.
The typed decoder overwrites an initialized byte array when the saved
array is all zero. With a struct containing one [1]byte field, save
{0}, then load it into {7}: the reflective decoder leaves {7}; the
typed decoder produces {0}. The typed decoder promises to preserve the
destination in this case, but its byte-array path calls ReadFull
unconditionally. The regression
uses the prototype’s unchanged production code. Only the typed,
initialized case failed.
For code size, keeping the varint loop out of line
puts a single copy in the wire package. In a matched AMD64 sentry
build,
.text shrank by 165,920 bytes; 142,573 of those bytes came from
1,240 generated StateEncode methods. Sixteen paired comparisons
on four-worker Nginx checkpoints
showed no detectable change in encode or decode time. The patch is
three lines directly on the prototype; the measurements used matched
qualification builds with the same change.
Posted by Codex on behalf of @tamird.
EtiennePerot4d agoSome autoresearch ideas for optimizing this:
| Idea | Decode/Encode CPU | Non-test delta | Description |
|---|---|---|---|
| Opt-in per-type stats | 12.4% / 10.7% | +67/-40 | state.Save/state.Load take a *Stats and skip the per-struct Nanotime timing when it is nil, and the kernel collects stats only when debug logging is on. |
| Narrow encode struct dedup | 0% / 16.5% | +47/-16 | The encoder replaces the encodedStructs map, which recorded every struct, with small maps that track only the two cases where a struct can be encoded twice (objects replaced by their container, and one boxed value shared by several interfaces). |
Lazy FileLocks lock sets | 7.4% / 8.4% | +41/-5 | vfs.FileLocks allocates its lock sets on first use, so the empty FileLocks of nearly every inode saves as one nil pointer instead of 9 nested struct values, and the state file is 4.3% smaller. |
| Zero-size struct fields as Nil | 3.9% / 5.1% | +3/-1 | Zero-size struct fields are saved as wire.Nil, so their type entry and their StateSave/StateLoad calls are skipped; old decoders can still read the new files, which are 2.7% smaller. |
| Single-field struct fast path | 5.1% / 16.2% | +77/-14 | go_stateify marks structs that save exactly one plain field (such as the atomicbitops wrappers), and pkg/state encodes and decodes that field directly; once stacked with the other findings it adds no measurable gain. |
| Cache interface itabs on decode | 9.2% / 0% | +31/-1 | Decoding an interface value stores a cached type word for each (interface, concrete type) pair instead of calling reflect.Value.Set, which runs the uncached reflect.implements method-set walk on every call. |
| Byte-reader and peeked varints | 10.0% / 0% | +192/-7 | wire.Reader reads through io.ByteReader (which the compressio readers now implement) and, during the object decode loop, parses whole varints from the underlying buffer instead of making one interface call per byte. |
Cached SaverLoader method dispatch | 3.5% / 3.5% | +75/-12 | Each type entry stores the type's SaverLoader interface value once, so encodeStruct/decodeStruct skip the Addr().Interface().(type) switch, which mostly fell through to getitab. |
Buffered wire.Writer output | 0% / 5.3% | +66/-25 | wire.Writer fills an internal 4 KiB buffer, flushed at the end of state.Save and by WriteHeader, instead of making one Write call per varint. |
I was expecting most of this to be dependent on the current implementation, but actually a bunch of them are actually pkg/state-implementation-details agnostic, such as:
- Narrow encode struct deduplication: Any implementation of
pkg/stateneeds to keep handle pointer identity and interior pointers. This is 28% of encoding CPU-seconds with the current implementation, so critical to look at for any implementation. - Buffered
wire.Writeroutput: Useful on its own irrespective ofpkg/stateimplementation details. - Byte-reader and peeked varints: Same as above.
- Opt-in per-type stats: Skip stats update is also independent of implementation.
- Zero-size struct fields: Code generation makes this easier because this can be known at build time and the code for encoding such structs can be not generated in the first place.
- Lazy
FileLockslock sets: Same direction as PR #16337, stacks up for locked files.
The other thing that's worth keeping in mind is that the //test/benchmarks/checkpointfile benchmark uses a trivial countingWriter, and that for more accurate measurements it should be swapped for the real statefile.NewWriter.
@tamird will you be working on this FR?