I've been having trouble with git being out of sync between host and containers in a Podman driven VM in my sandbox library foldyard and had to write pretty elaborate git shims, monitors and heal scripts. What led me to this bug is that with Lima / vz the issue was nowhere near as bad because the time windows are much shorter.
Questions:
- When a path-based operation on a tracked, non-root inode fails with ENOENT because its
/.vol/{dev}/{ino}no longer resolves (and it has nounlinked_fd), would returning ESTALE be acceptable? The guest would then re-look-up the name straight away. It would find the new file if one exists, and get a real ENOENT if the file was deleted. NFS clients rely on the same semantics. - Failing that, would you take a patch that makes
entry_timeoutconfigurable (libkrun API plus a krunkitvirtio-fsoption), so that users sharing a tree the host also writes could lower it?
Claude's detailed writeup:
Summary
On a macOS host, when the host atomically replaces a file in a shared directory (write a temp
file, then rename() it over the existing name), a Linux guest that has already looked up that name
gets ENOENT for it until the guest's dentry expires, which takes up to 5 seconds. New names show
up at once, and writes from the guest reach the host at once.
git updates HEAD, loose refs, packed-refs and .git/index in exactly this way. So git in the
guest sees a branch that the host just moved as missing. It falls back to packed-refs (an older
commit) or treats the branch as unborn, and a guest commit made in that window can rewind the branch.
Environment
- macOS 26.7, Apple Silicon
- podman 6.0.2 → krunkit 1.3.2 (
/opt/podman/bin/krunkit) → the libkrun it bundles (≥ 1.19.3; it exportskrun_add_virtiofs4) - Guest: podman machine's Fedora CoreOS 44.20260621.3.1, kernel 7.0.12-201.fc44.aarch64
- The share is podman's default
virtio-fs,sharedDir=…,mountTag=…with nopermissionSemantics, so krunkit's defaultsimplifiedapplies (attr_timeout= 0,entry_timeout= 5 s)
Reproducer
Host (macOS), inside a directory the machine mounts, for example under $HOME:
# host.py: replace ./f with a new version every 2 s
import os, time
for i in range(30):
with open("f.tmp", "w") as fh:
fh.write(str(i))
os.replace("f.tmp", "f")
time.sleep(2)Guest (podman machine ssh, same path):
# guest.py: stat ./f every 10 ms, print each ENOENT window
import os, time
t0, gone = time.monotonic(), None
while time.monotonic() - t0 < 60:
now = time.monotonic() - t0
try:
os.stat("f")
if gone is not None:
print(f"ENOENT {gone:7.3f}s .. {now:7.3f}s ({(now - gone) * 1000:.0f} ms)")
gone = None
except FileNotFoundError:
gone = now if gone is None else gone
time.sleep(0.01)Start guest.py, then host.py. Replaces then produce ENOENT windows in the guest.
Observed
- ENOENT windows of 0.2–4.7 s after each host replace. The new version appears at t ≈ 5, 15, 20, 30, 35 s, which is a 5 s grid rather than a fixed delay after each replace.
- For comparison, on the same Mac, Apple's Virtualization.framework virtio-fs (Lima
vz, and podman'sapplehvprovider) shows the same shape but for only 22 ms to ~1 s. - A
link()-family call on the name (for exampleln -dT / f, which fails with EEXIST and creates nothing) makes the next read see the new file at once, for every process in the VM.
What the source suggests
Please correct me if I've misread any of this.
macos/passthrough.rsaddresses a node by its volfs path/.vol/{dev}/{ino}(inode_to_handle). Once the host'srename()drops the old inode's last link, that path no longer resolves.do_getattr/open_inodethen pass the host's ENOENT to the guest. #700 fixed this for renames the guest makes, throughunlinked_fd. A rename by the host isn't visible to libkrun, so the fix can't cover it.- The guest kernel treats ENOENT from GETATTR/OPEN as final for as long as the dentry is valid. The
dentry lifetime is
entry_timeout, which isConfig::default()'s 5 s.Fs::newoverrides onlyattr_timeout. As far as I can see, neither value is exposed through the C API or krunkit. - The
lnworkaround works becauseLOOKUP_EXCLforcesfuse_dentry_revalidateto re-issue LOOKUP.LOOKUP_REVALis in the same mask. The VFS retries a path walk withLOOKUP_REVALwhen an operation returns ESTALE (vfs_statx'sretry_estale, and the same indo_filp_open).
Activity
0No activity yet.