Takes the host sync of a FLUSH off the virtio-blk worker for raw images: a per-disk flush thread runs one sync at a time and completes every FLUSH it took before starting it, so reads and writes keep being served during a sync and concurrent FLUSH requests share one. The commit message has the rationale, the soundness argument, and the measurements.
Full analysis (what F_FULLFSYNC costs on macOS and where the time goes, the invariants and how each is tested, method and all results, trade-offs, open questions) and raw data: https://gist.github.com/ArtBlnd/3537b82d4601ce92395b035f91f96ce1
Related to #824: this takes the sync off the worker, but reads and writes are still served one at a time. The change also applies to stable-1.19.x (cherry-pick-1.19.x), where it additionally needs 137f84bd.
I've tried to investigate why Apple SSD stalls. Still found nothing.
There is two assumptions: