#2367

sync --apply still deletes drawers for files hidden under a mount, in the shapes #2322 left open

Openmvalentsevopened 2mo ago

What happened?

sync --apply still removes drawers whose source file it could not reach, in three mount shapes. They are one gap: the neighbour that corroborates the removal is not in the directory the missing file was mined from, and nothing in the palace records which directory that was.

Measured on develop @ 4c1e6d0c, which carries #2322, with real mount and umount inside unshare --mount --map-root-user, through the real CLI. One run each, and in each one mempalace search stops naming a file that is on disk the whole time.

shapesync --apply runsreportswhat was removed
B, a volume mounted over a directory the palace knowswhile it is mountedMissing: 1, Unresolved: 0the drawers of x.md, which is underneath the mount
A, a mined file in the mount point's lower layerafter the unmountMissing: 2, Unresolved: 0the drawers of the volume's two files, corroborated by the x.md underneath that outlived the unmount. Remount and both are back without their drawers
C, a bind mount over a directory the palace knowswhile it is in placeMissing: 1, Unresolved: 0the drawers of keep.md, which is underneath the bind mount

Shape C is what docker -v does: measured, a -v mount shows up in the container's mountinfo as a bind mount of that host subtree, and the directory answers with the host's own inode. The README's own recipes mine a source that exists only as such a mount (README.md:101, :123).

What did you expect?

A removal to require that the corroborating neighbour is in the directory the missing file was mined from, decided without writing into the source tree, since README.md:99 promises mining never does.

Neither the path nor the identifiers a filesystem hands out can decide it. Measured here, one run each:

  • st_dev for tmpfs and other anonymous filesystems comes from a pool in mount order: three mounted in turn answered 132, 133, 134, and in the reverse order 134, 133, 132. For a filesystem on a block device it is that device's major:minor instead, stable but identical on both sides of a bind mount.
  • f_fsid is the filesystem's own UUID folded in half rather than an identifier of what is mounted: for / it is 8737877634232224924, the XOR of the two little-endian halves of /dev/sdd's UUID. It is 1 on the Windows drives a WSL install exposes.
  • The directory's own inode answered 1331592, then 1 under a tmpfs mounted over it, then 1331592 again after the unmount, and the sibling's inode under a bind mount of that sibling. It belongs to the filesystem's structure rather than to this boot's mount table, so it changes exactly when the name starts resolving to something else.

How to reproduce:

Inside unshare --mount --map-root-user, with <p> a scratch palace and prose any text long enough to file a drawer. A and B share one setup and differ only in when the sync runs.

  1. mkdir -p proj vol && prose > proj/x.md
  2. mempalace --palace <p> mine proj --wing w
  3. mount --bind vol proj
  4. prose > proj/y.md && mempalace --palace <p> mine proj --wing w
  5. Shape B: mempalace --palace <p> sync proj --wing w --apply reports Missing: 1 (removed). Then umount proj: x.md is still there, its drawers are not.
  6. Shape A: on a fresh palace, repeat 1 to 4 with a second file mined from the volume, then umount proj before the sync. It reports Missing: 2 (removed), and remounting brings both files back without their drawers.

Shape C needs no volume: mkdir -p proj/sub other && prose > proj/sub/keep.md, mine, mount --bind other proj/sub, add proj/sub/o.md and mine again, and the sync reports Missing: 1 (removed) for a keep.md that never left the disk.

Environment:

  • OS: Linux (WSL2)
  • Python version: 3.12.3
  • MemPal version: develop @ 4c1e6d0c

Notes

A re-mine brings a removed drawer back once the file is reachable again, since file_already_mined returns False for a source with no drawers left. What it does not bring back is anything the miner did not write. Measured, one drawer filed by mempalace_add_drawer against that source: 1 before the sync, 0 after it, 0 after a re-mine that refiled the miner's own.

#2320 is the report these were carved out of, and #2322 already names shape A as staying open, with the reason: "separating those needs the filesystem identity of each source at mine time, which nothing records". B and C are not named there and remove drawers on develop today as well. The one other shape #2322 leaves open is none of these three: a volume that leaves and returns between two adjacent stat calls.

Activity

0

No activity yet.