serialize() and from_serialized(): a file format for maps of trivially copyable Key and T, once a caller asks

The number

None. Do not start this until a caller asks for it. Split out of #299 for that reason:

• The four parties who built a load path against this map (drecouse's fork, motis-project's fork, #138, #19) each embed the map in a format of their own or want a view. None of them asked for a file format from this library, and #299's raw layer (values(), index(), a constructor from both) serves all four.
• A published header with a magic number is a persistent format. The index changed its layout twice in September 2026 (groups of sixteen on 09-05, the value indices merged into the group's block on 09-06, 8472a5d) and the string hash once (219f0dc, 09-06). "No migration, a mismatch means rebuild" is the policy, but every such change is then a release note that tells file owners to rebuild.

What would open it: a caller with Key and T trivially copyable whose map is the whole file, who would otherwise write the thirty lines of framing (header struct, alignment, format check, bounds) themselves.

What to build

A framing layer on top of #299's raw layer, for the case where Key and T are trivially copyable and the map is the whole file.

// Enabled when std::is_trivially_copyable_v<Key> and T is void or trivially copyable, checked on
// Key and T separately, never on the pair (std::pair is not trivially copyable). Calls
// sink(void const* p, std::size_t n) for each piece in order: the 64-byte header, padding, the
// values, padding, the index. No staging buffer: a 4 GB map goes to pwrite() piece by piece.
template <typename Sink> void serialize(Sink&& sink) const;
[[nodiscard]] auto serialized_size() const noexcept -> std::size_t;

// Checks magic, header version, index_format_id, the value layout fields, that every
// offset and count fits inside `size` with overflow-safe arithmetic, and that both arrays land on
// their alignment. Then #299's matching constructor: the view flavour holds the two arrays in
// place with `t`, the owning flavour copies them and always checks. Failure goes through the
// same on_error_* hook. Nothing on this path hashes and nothing calls verify(); that is the
// caller's call, it hashes.
static auto from_serialized(std::byte const* src, std::size_t size, trust t,
                            Hash const& hash = Hash{}, KeyEqual const& equal = KeyEqual{}) -> table;

The header is 64 bytes, static_asserted for size and has_unique_object_representations_v:

• the 8-byte magic {0x89, 'U', 'D', 'M', '\r', '\n', 0x1a, '\n'} (PNG's structure: the high bit catches a 7-bit channel, CRLF then LF catch a text-mode conversion in either direction, Ctrl-Z stops DOS type, UDM is what xxd shows a human);
• uint32_t header version, 4 reserved bytes;
• uint64_t index_format_id (#299's, which covers the index and the hash and deliberately not the values);
• uint16_t sizeof(Key), sizeof(T) (0 for a set), sizeof(value_type), and the offset of .second in value_type (0 for a set), because index_format_id does not cover the values. The offset is the address difference of .first and .second in a value-initialised value_type, since offsetof on std::pair is only conditionally supported;
• uint64_t num_values, num_blocks, values_offset, index_offset.

8 + 8 + 8 + 8 + 32 = 64. No total_size field: the end is index_offset + num_blocks * sizeof(index_block), computed with the same overflow-safe arithmetic as the other bounds and checked against size. static_assert(sizeof(value_type) <= 65535) on serialize(), so none of the four fields can truncate.

Offsets are multiples of 64 from the start of the header, which satisfies both alignof(index_block) values and any value type up to a cache line, provided the blob itself starts 64-aligned (a mapping does, a std::vector<std::byte> may not; say so). static_assert(alignof(Key) <= 64 && alignof(T) <= 64) on serialize(), so an over-aligned type is a compile error rather than a misaligned file. Alignment is not in the file. It is a property of the reader, not of the bytes: the reader checks the two array pointers against its own alignof(value_type) and alignof(index_block), and the 64-byte offsets satisfy both on every target. Recording alignof would reject files whose bytes are identical: std::pair<uint64_t, uint64_t> is 16 bytes with .second at 8 on both i386 and x86-64, but its alignof is 4 on one and 8 on the other (measured, g++ -m32 and -m64). A layout that does differ is caught by the size fields: std::pair<uint32_t, uint64_t> is 12 bytes on i386 and 16 on x86-64.

from_serialized() rejects a header version it does not know; no migration, no reading of older formats.

serialize() on a segmented_map emits the segments as consecutive pieces, so the file has the same flat layout as from a map; from_serialized() on a segmented_map copies into segments; a map_view opens a file a segmented_map wrote.

serialize() does not exist for values that are not trivially copyable, and the error message says to use values() and index().

The docs recommend rehash(size()) before serialize(): a view never runs move_home, so a churned table's drift (1.14 groups per hit against 1.03 fresh) is frozen into the file.

The README rule, in the nonstandard API section: Key and T trivially copyable and the map is the whole file, serialize() and from_serialized(); anything else, #299's raw layer.

How to measure it

serialize() into a file and from_serialized() back at 1M, 4M, 16M and 64M entries of map<uint64_t, uint64_t>, against #299's raw layer on the same bytes: the framing's checks must be O(1) and read as noise.

Done means

• The three members above, the header struct with its static_asserts, the README's section with the header laid out field by field.
• Tests: serialize() into a 64-aligned std::vector<std::byte> and from_serialized() back, both flavours, both bucket types, map and set, empty map included, agrees on every key, and the bytes delivered equal serialized_size(). Rejected, each with a mutation (scripts/mutate/mutate.py --diff): a header from a group_big map fed to a group view, a wrong magic, an unknown header version, a wrong sizeof(T), a wrong sizeof(value_type), a wrong offset of .second, a truncated buffer, an offset past size, a count that overflows the multiplication, a misaligned src.
• A file of map<uint64_t, uint64_t> written on x86-64, checked in under data/index_format/, opened on every leg: on the -m32 legs it must open and agree on every key (same bytes, different alignof), which is the test that alignment stayed out of the header. Skipped on big endian.
• A second entry point in #299's fuzz_index_view: random bytes into from_serialized(). Either it rejects, or a fixed sequence of lookups completes without a sanitizer report.
• -fno-exceptions, 32-bit, ASan+UBSan and unity legs reproduced locally before pushing.

Blocked on #299.