This adds locked guest memory: memory on top of a VM's RAM that libkrun locks into host memory with mlock(), and that the guest sees as a NUMA node of its own, without CPUs. It's VmmBuilder::locked_memory_mib(), and krun_vmm_builder_locked_memory_mib() in C.
Today a guest has no way to keep a process resident. When the host is short of memory, macOS compresses and swaps out guest RAM like any other memory of the VMM, and with free page reporting, the pages that the guest frees go back to the host and fault in again on their next use. Some guest processes need to stay responsive whatever the host does, for example the agent daemon that we run in each guest. With locked memory, the guest binds such a process to node 1 (mbind(), numactl --membind=1), and the host keeps its pages resident. It's a region of its own rather than all of RAM, so the rest of the guest's memory is still compressed, swapped and given back through free page reporting as before.
Based on #955 (vmm/macos: build the vstate tests again), without which the tests of the libkrun crate don't build on macOS.
The commits:
- lib reports start errors with Display instead of Debug, so that the error of 3 reaches the caller with the limits it names.
- arch/aarch64 puts the region at the first 1 GiB boundary above RAM, where SHM starts today, and moves SHM up. RAM, the FDT and
ram_last_addrdon't move, so checks for RAM, like the one inVmm::reclaim_memory(), keep the region out. - vmm locks it before the hypervisor maps it, and refuses to start the VM if that fails.
- fdt/aarch64 describes it as NUMA node 1.
- virtio/balloon keeps free page reports in it away from madvise() and from the Replace mode, whose MAP_FIXED remapping would silently unlock it on macOS.
- lib adds the builder method and regenerates the bindings.
A size of 0, the default, changes nothing: the layout and the FDT are the same as before. Only aarch64 supports it for now (x86_64 would also need an SRAT), so elsewhere a size other than 0 fails with FeatureDisabled, as pause() does where it isn't supported.
Testing:
- Unit tests on macOS (aarch64) and on Linux (aarch64, in a VM without
/dev/kvm, so without the tests that need KVM). With a size of 0, the tests check that the FDT has no NUMA information and that its memory node is unchanged; with locked memory, that only the new memory node, the numa-node-id properties and the distance map are added. Taking the trimming out ofprocess_frq()makes the balloon tests fail, and makingmlock()fail, or not calling it, makes the lock tests fail. - Every commit passes
cargo fmt --checkand clippy with-D warningson macOS, and for aarch64 Linux, x86_64 Linux and x86_64 Windows (cross-checked from macOS, with-A deprecatedbecause virtio-rtc uses the deprecatedlibc::time_ton musl), andmake gen-libkrun-bindingsleaves no diff. - End to end, with the same change on 1.19 (krunkit and Lima on macOS, EFI boot, 1 GiB of RAM and 256 MiB of locked memory): the VMM's wired memory was 256 MiB at every sample, and vmmap showed 239 MiB of the RAM mapping compressed or swapped out and none of the locked mapping; the guest saw node 1 with 247 MB and no CPUs, at distances of 10 and 20 (
numactl -H); the balloon skipped 686 MiB of free page reports in the locked region and kept giving back free RAM. - On main, I only got as far as the VMM:
examples/boot_efi.cwithkrun_vmm_builder_locked_memory_mib(&builder, 256, ...)loggedlocked guest memory: 256 MiB locked (guest 0xc0000000-0xd0000000), and the VMM's resident memory was 326 MiB, against 70 MiB without it. I couldn't boot a guest there to see node 1 and the balloon: the EFI firmware that krunkit uses doesn't get past its first steps on main in my setup, with or without this change (also on unmodified main). Those two results are from 1.19.
Once this is merged, I'll send the same series to stable-1.19.x, as #931 was backported in #941.
🤖 Generated with Claude Code