What happened?
sync --apply still removes drawers whose source file it could not reach, in three mount shapes. They are one gap: the neighbour that corroborates the removal is not in the directory the missing file was mined from, and nothing in the palace records which directory that was.
Measured on develop @ 4c1e6d0c, which carries #2322, with real mount and umount inside unshare --mount --map-root-user, through the real CLI. One run each, and in each one mempalace search stops naming a file that is on disk the whole time.
| shape | sync --apply runs | reports | what was removed |
|---|---|---|---|
| B, a volume mounted over a directory the palace knows | while it is mounted | Missing: 1, Unresolved: 0 | the drawers of x.md, which is underneath the mount |
| A, a mined file in the mount point's lower layer | after the unmount | Missing: 2, Unresolved: 0 | the drawers of the volume's two files, corroborated by the x.md underneath that outlived the unmount. Remount and both are back without their drawers |
| C, a bind mount over a directory the palace knows | while it is in place | Missing: 1, Unresolved: 0 | the drawers of keep.md, which is underneath the bind mount |
Shape C is what docker -v does: measured, a -v mount shows up in the container's mountinfo as a bind mount of that host subtree, and the directory answers with the host's own inode. The README's own recipes mine a source that exists only as such a mount (README.md:101, :123).
What did you expect?
A removal to require that the corroborating neighbour is in the directory the missing file was mined from, decided without writing into the source tree, since README.md:99 promises mining never does.
Neither the path nor the identifiers a filesystem hands out can decide it. Measured here, one run each:
st_devfor tmpfs and other anonymous filesystems comes from a pool in mount order: three mounted in turn answered132,133,134, and in the reverse order134,133,132. For a filesystem on a block device it is that device'smajor:minorinstead, stable but identical on both sides of a bind mount.f_fsidis the filesystem's own UUID folded in half rather than an identifier of what is mounted: for/it is8737877634232224924, the XOR of the two little-endian halves of/dev/sdd's UUID. It is1on the Windows drives a WSL install exposes.- The directory's own inode answered
1331592, then1under a tmpfs mounted over it, then1331592again after the unmount, and the sibling's inode under a bind mount of that sibling. It belongs to the filesystem's structure rather than to this boot's mount table, so it changes exactly when the name starts resolving to something else.
How to reproduce:
Inside unshare --mount --map-root-user, with <p> a scratch palace and prose any text long enough to file a drawer. A and B share one setup and differ only in when the sync runs.
mkdir -p proj vol && prose > proj/x.mdmempalace --palace <p> mine proj --wing wmount --bind vol projprose > proj/y.md && mempalace --palace <p> mine proj --wing w- Shape B:
mempalace --palace <p> sync proj --wing w --applyreportsMissing: 1 (removed). Thenumount proj:x.mdis still there, its drawers are not. - Shape A: on a fresh palace, repeat 1 to 4 with a second file mined from the volume, then
umount projbefore the sync. It reportsMissing: 2 (removed), and remounting brings both files back without their drawers.
Shape C needs no volume: mkdir -p proj/sub other && prose > proj/sub/keep.md, mine, mount --bind other proj/sub, add proj/sub/o.md and mine again, and the sync reports Missing: 1 (removed) for a keep.md that never left the disk.
Environment:
- OS: Linux (WSL2)
- Python version: 3.12.3
- MemPal version:
develop@ 4c1e6d0c
Notes
A re-mine brings a removed drawer back once the file is reachable again, since file_already_mined returns False for a source with no drawers left. What it does not bring back is anything the miner did not write. Measured, one drawer filed by mempalace_add_drawer against that source: 1 before the sync, 0 after it, 0 after a re-mine that refiled the miner's own.
#2320 is the report these were carved out of, and #2322 already names shape A as staying open, with the reason: "separating those needs the filesystem identity of each source at mine time, which nothing records". B and C are not named there and remove drawers on develop today as well. The one other shape #2322 leaves open is none of these three: a volume that leaves and returns between two adjacent stat calls.
Activity
0No activity yet.