#16209

Optimization: Read local state files through the async read-ahead reader during restore

Openmilantracyopened 8d ago

The idea

For local checkpoint files, restore decodes the state file through a 256 KiB bufio.Reader on the raw FD, so each refill is a synchronous read on the decode path. During --background restore, these reads wait behind the page loader's reads on the same disk. The checkpoint-gofer path already uses stateio.NewBufReader with an 8 MiB buffer that reads ahead. The local path can use the same reader with stateio.NewFDReader.

In getRestoreReadersForLocalCheckpointFiles, if the state file is a regular file, wrap it as stateio.NewBufReader(stateio.NewFDReader(fd, ...), 8<<20). Keep the current path for non-regular files (FIFOs, #15553). Use a small maxParallel, because aio.GoQueue raises GOMAXPROCS (#16138).

Evidence

  • runsc/boot/controller.go:496-533: the local path returns the raw state file FD.
  • pkg/compressio/nocompressio.go:108-116: bufio.NewReaderSize(in, 256*1024).
  • runsc/boot/controller.go:551-558: the gofer path uses stateio.NewBufReader(..., 8<<20).

Performance impact

Not measured yet. Related: in #15880 (n=32, benchstat), throttling background page reads until the kernel restore is complete made the restore critical path, which includes this decode, 89–90% shorter with a cold page cache. So state file reads are sensitive to I/O queueing.

How to verify

Measure "Kernel load took" and runsc restore time with a cold page cache on a workload with a large state file (many tasks or files), to significance (n=32, benchstat). Run the C/R container tests.

Activity

0

No activity yet.