#16676

runsc: Destroy fails with ENOENT on the rootfs filestore under CRI-O

Openfivetimeopened 1d ago

Description

Under CRI-O, the first RemovePodSandbox of a gVisor pod fails:

failed to delete container k8s_test_gvisor-test_default_gvisor-test_0 in pod sandbox f54bd38a...: failed to delete task: destroying container: failed to delete filestore file "/var/lib/containers/storage/overlay/18be61ea.../merged/.gvisor.filestore.f54bd38a...": remove /var/lib/containers/storage/overlay/18be61ea.../merged/.gvisor.filestore.f54bd38a...: no such file or directory: unknown

CRI-O unmounts the app container's rootfs when it stops the container, so the rootfs filestore is gone by the time the shim destroys the container. Container.Destroy treats that ENOENT as an error for the container's own mounts:

https://github.com/google/gvisor/blob/0fa6803f2a5991dc066a29fc50414b4a3b519590/runsc/container/container.go#L1147-L1151

while a few lines later, for the filestores of shared mounts, it ignores ENOENT:

https://github.com/google/gvisor/blob/0fa6803f2a5991dc066a29fc50414b4a3b519590/runsc/container/container.go#L1163

The container's state files are deleted before this point, so a second RemovePodSandbox succeeds, and kubelet's retry hides the problem. It still shows as an error in the CRI-O log and as a failed crictl rmp.

Expected: Destroy ignores ENOENT for the container's own filestores too (err != nil && !os.IsNotExist(err)), as it does for shared mounts.

Steps to reproduce

The setup is the one in g3doc/user_guide/containerd/crio.md: CRI-O 1.37.2, containerd-shim-runsc-v1 as a runtime_type = "vm" runtime, grouping = true in its runsc.toml.

pod=$(sudo crictl runp --runtime gvisor pod.json)
ctr=$(sudo crictl create "$pod" ctr.json pod.json)
sudo crictl start "$ctr"
sudo crictl stopp "$pod"
sudo crictl rmp "$pod"    # fails as above
sudo crictl rmp "$pod"    # succeeds

with e.g. a busybox container running sleep 600.

runsc version

release-20261019.0; the code is unchanged on master (0fa6803f2a)

uname

GitHub Actions ubuntu-24.04 runner

kubectl (if using Kubernetes)

Reproduced with crictl against CRI-O 1.37.2; Kubernetes not needed.

Activity

0

No activity yet.