#3197

Keep a record's key order per value where no table or order index holds it

Draftcramforcewants to merge intofrom

A compiled program printed a record's keys in a different order from Node, with no diagnostic, whenever the record's type had neither one key order nor a small countable set of them. Object.keys, for…in, JSON.stringify, spread, Object.entries and console.log all showed it.

interface Base { id: string; name: string }
interface Item extends Base { ready: boolean; limit?: number }
const loaded = JSON.parse(text) as Item[];
const all = loaded.map((item) => ({ ...item, name: item.name + "!" }));
const mine: Item = { id: "a", name: "A", ready: true };   // Node: id,name,ready   main: ready,id,name

Every type that is converted from a dynamic value and then cloned or copied fell back to its declaration order for every value, which for an interface that extends a base lists the derived interface's own keys first. That is most data types of a program that parses JSON and copies what it parsed. A natively built pi writes its session files and RPC answers that way.

The rule

One invariant decides everything per value: a record that keeps no kept-source entry, and is no copy linked to the record it was narrowed from, holds exactly its declared keys in its type's table order. Every record in another order has an entry.

  • The table is chosen from the constructions. A literal contributes the order it writes. A copy (rest binding, record conversion), a spread and a clone contribute their source type's table, also when the source type is converted from dynamic values: a source without an entry is in that order, and a source with an entry hands its key list on when the construction runs. "Converted and cloned" and "copied from a converted type" are no longer reasons to give up on a type.
  • A type is tracked only when its constructions disagree (two literal orders, a spread at two positions, optional keys from two sources). Its table is then the order the most construction sites write, with the keys that writes create listed last in first-write order, so a value built in the common way keeps nothing; the other constructions note their own order, and a write or clone that creates a key moves it in the value's own list.
  • Per value, at the site. A copy makes one call, record.orderCarry, which returns at once for a source without an entry and builds no view of it. A spread literal tests record.sourceHas per spread value when the spread runs and snapshots only a value that has an entry; for the others the literal's order is known at compile time, so it notes nothing when that order is the table's, and otherwise notes placeholders, testing presence only for a key a later property can also create. A clone keeps #3163's test. A write that creates a key tests presence first.
  • A value built in its type's order never allocates a key list, a boxed view or a table entry.

Also in this PR:

  • Keyed writes. r[k] = v on a closed record with optional keys (Partial<Record<Token, V>>) was not counted as a write that creates a key, so a record filled by key listed its keys in slot order. Such a type is tracked, and the write tests the key and moves it last in the record's own list.
  • Linked copies. A value converted to a narrower record type is a copy linked to its source, and its keys are the source's. The per-value test counts the link (record.linked), in spreads, copies and clones, and the source type's table takes the order the source was built in. On main a narrowed copy lists the source's keys in declaration order and a clone of it drops the source's other keys.
  • Object.assign and dictionary spreads copied the source's fields in slot order, so the keys a target gained came out sorted by name. They are created in the source's own key order.
  • A record whose checked view already exists when a write first creates a key out of table order: the view adopts the record's key list.
  • Runtime. Four table operations in a unit of their own, scr_record_order.c, linked only by programs with a tracked or carried type. scr_json.c is not edited.
  • SC6002 from the lowering's own facts. An observed type left in one order although its values are created in several, at the construction whose keys the pass could not read. New: an index-signature record with declared properties lists the declared ones first, and SC6002 now reports a write or literal that creates a declared property after another key, and an integer-like key written by name, when a key walk or a serializer reads the type and a value of it can hold an undeclared key. tests/corpus/width-layout-index-signature.ts does exactly that and prints only serialized lengths; it names the warning on its first line (// @divergence-warnings: SC6002, read by coverage.test.ts). The exact fix for those records is its own change. The older syntactic warning ("a property this object leaves out is assigned later") no longer fires for a literal whose type lowers to checked-dynamic objects (a member typed object or unknown), which keep their keys in creation order; tests/coverage-fixtures/no-divergences.ts pins it.

Shapes from real programs

Neutral reductions, each under Node 24.22.1 and natively:

shapemainthis branch before this revisionnow
entries of a union of interfaces extending one base, parsed and cloned (pi's session entries: type, level, id, parentId, timestamp for type, id, parentId, timestamp, level)differsmatchesmatches
a record extending a base, built from a table, parsed and cloned (pi's model records)differsmatchesmatches
{ k: v, ...recOrNull } where the program builds the type in another order elsewherediffersmatchesmatches
a literal out of declared order with a conditional spread (the compiler's FFI loader)differsmatchesmatches
Partial<Record<Token, V>> filled by key, spread, filled again (pi's export CSS: 56 variables in alphabetical order)differsdiffersmatches
a narrowed copy, and a clone of itdiffersdiffersmatches
a tracked literal spreading a value whose own type nothing enumerates (the review bot's first finding)differsdiffersmatches
a type that a rest copy produces and a literal also builds in another order, copied from a parsed or cloned value (the review bot's second finding)differs, with SC6002differsmatches
a type built only by rest copies of a type whose literals disagree, copied from a parsed value (found by the generator's new mode)differsdiffersmatches

The last two are one class: a copy of a value that keeps its own key list was given an order index (its source type's table, or the source's own index) instead of the list. A type with such a copy takes no index now; its copies read the source's keys when they run.

Proof

All runs in Vercel Sandboxes; Node 24.22.1.

Generated programs. gen-rec.mjs, 80 programs per seed, seeds 7, 11, 12, 13: 320 of 320 match Node (main: 129 match, 32 warned, 159 silent). With its new --rest mode (a second type that a rest copy produces and literals also build in random orders, and values parsed from JSON text in random key orders), seeds 7, 11, 12, 13, 21, 22, 23, 24: 640 of 640 match; main matches 57 of the first 320, with 18 warned and 245 silent. The generative lane of #3168 (generate, seed 1, 600 programs): 112 silent differences on main, none here.

Probe matrix. 338 programs: 287 match (main 229 of 337). The rest are 23 refusals and 28 index-signature programs, unchanged.

Corpus. 8751-key-order-carried.ts (new), 8922-key-order-tracked-literals.ts, 8923-key-order-tracked-writes.ts, 8918-key-order-object-assign.ts, 8919-key-order-tracked-copies.ts. Unit tests in key-order-witness.test.ts.

Zero-cost gate, --gate-corpus=all, allow pattern \{[^{}]*:|\.\.\., head f1c7344a19 against main b3ff41013c, which it contains: PASS. 5,282 programs identical, 118 fail to build on both sides, 32 differ, no runtime-only row. Every benchmark is byte-identical (bench geomean +0.79%, CI -0.40% to +1.59%; bench-runtime 65 neutral, 0 faster, 1 slower: alloc-trees +4.1% on an identical executable). The round before it, 4cc954f0ce against 50bcfe1b3, had the same result on 30 programs.

The 32 that differ each hold a record type this change tracks, carries or copies in key order. The first commit of this PR changed 22; this revision adds eight and drops none, because types that derive from converted or narrowed values now carry the value's order where the first commit left them in one order; two more are programs new on main:

  • Dictionary spreads and Object.assign in source key order: 1555-index-spread-into-declared (+9.69%, the largest growth: the first program there to link the dynamic key walk), declared-index-spread, spread-union-declared-merge, record-presence-copies.
  • Spreads at several positions or of union-typed values: 2691-retained-lowering-record-order, record-union-spreads, union-spread-distribution, 5230-inline-union-fields, key-order-spread-copies (new).
  • A literal in another order than the runtime's own records of the type: 5047-generator-delegation-iterables-typed.
  • Copies of values whose order is only known at run time: 6471-union-literal-results-into-record, 9211-object-entries-declared-union, checked-union-record-assertions, localized-failure-pi-shapes, localized-failure-expressions (new), 7623-key-order-javascript-built-records (new), 7624-key-order-boxed-callback-records (new), 2470-mockable-module-shape.js (new, +128 bytes), 7614-key-order-copies (new, -1.11%: a type that mirrored its source's order index and now reads the source's keys).
  • Narrowed copies: width-layout-narrowed-arm (new), 8822-asserted-optional-chain-member-read (new).
  • A write that creates a key on a parsed or kept record, or after delete: json-typed-parse-optional-members, record-presence-delete-json, 4412-nullable-record-values, 4930-typed-record-reflection, width-spread-kept-keys, width-spread-kept-keys-delete.
  • New on main since the round before: 8876-partial-assertion-completed-first (+5.94%), 8962-asserted-subtype-record-beside-undefined (+4.57%).
  • The compiler itself: self/cli, fixture/llvm-emitter, fixture/llvm-layouts.

Sizes change by -1.11% to +9.69%; the gate ran with --gate-max-size-growth=15.

Lanes, bootstraps, Test262 (the same round, head f1c7344a19 against b3ff41013c): Test262 0 regressed in Object, Reflect, JSON, Proxy and Function. Both native bootstraps pass, plain and sanitized. Plain lane 22,489 passed, 1 failed; sanitized 19,233 passed, 1 failed. Neither is in this change, and each passes when run alone on this head: native-toolchain.test.ts "TLS builds promote staged vendor archives" is a known timing flake (65 s alone), and effect4.test.ts cluster-entity.ts had its compile step stopped at the 540 s limit in a round whose machines were about 40% slower than the round before (236 s alone, sanitized). The round before, 4cc954f0ce against 50bcfe1b3: plain 22,329 passed and sanitized 19,074 passed, with the six failures per lane that main had then (executable-cache.test.ts, five island-value-matrix cells) and has fixed since.

Cost on the natively built compiler. This is why the PR is a draft: the bar is the compiler compiling itself within +5% time and +6% memory, and the time is not within it. Both native compilers are built in one Sandbox and run interleaved, 32 vCPU Intel Xeon 2.90 GHz.

revisionbasetime, per pairpeak memory, per pair
the first commit of this PR (Xeon 2.50 GHz)235d40485+16.3%, +3.0%, +13.1%+28.7%, +25.9%, +26.4%
the head first pushed as this revision (250dafbc98)dfa7753d3+6.5%, +9.8%, +4.6%+6.9%, +10.0%, +5.6%
with "a late write's key list names only the keys the record has"dfa7753d3-1.1%, +5.1%, +7.4%+1.6%, +3.5%, +2.3%
4cc954f0ce (this head without the merge of main), four pairs, two in each order50bcfe1b3+3.9%, +13.7%, +15.2%, +8.6% (base 394.9, 386.1, 384.0, 396.8 s; branch 410.4, 438.9, 442.4, 430.8 s)-1.1%, +1.5%, +0.8%, -0.3% (base 7.07, 6.90, 6.93, 6.95 GB; branch 6.99, 7.01, 6.99, 6.93 GB)
the same two binaries and the second row's code on this base (936c4ce97b), three rounds of all three50bcfe1b3936c4ce97b: +11.9%, +11.6%, +12.8%; 4cc954f0ce: +7.8%, +13.4%, +9.7% (base 387.6, 442.0, 397.5 s)both -1.4% in every round (base 7.08 GB)

Memory is at the base's level on the current base. Time is not: 4cc954f0ce is +10.3% in the mean of its seven pairs against 50bcfe1b3, and the code first pushed is +12.1% there, where it measured +7.0% against dfa7753d3: main moved under it. The runs of one binary differ by up to 8% on that machine, so the pairs do not agree with each other, but none of the seven is a gain. A census of the compiler's own build shows the same decisions as before the last two commits (460 tracked types, 91 carried, 250 literal sites that write a tracked type's keys in a minority order), so those commits did not move the cost. The two fixtures on the last machine were too noisy to quote for time (1.17 to 1.87 s for the same binary and input); their peak memory is steady: ir-traversal.ts 143.1 to 146.8 MB (+2.6%), integer-ranges.ts 232.9 to 238.1 MB (+2.2%). On a quieter machine, with the second row's head: ir-traversal.ts 1.219 to 1.186 s, integer-ranges.ts 4.311 to 4.357 s (7 pairs each).

The compiler's own LLVM text grows 1.76% and its binary from 112.3 to 115.4 MB. With the Node-run compiler the gate's self figure is +0.98% (CI +0.84% to +1.01%).

What is left, from a sampling profile of both binaries on the current pair (50bcfe1b3 and 4cc954f0ce; 491 s and 533 s of sampled CPU time, +8.4%): the dispose hook, scr_dispose_observers, +8.8 s (3.0% of the base's samples, 4.4% of the branch's), and the out-of-line record.sourceHas call, +8.7 s (0.2% to 1.9%). Together they are 17.5 s of the 41 s; scr_dyn_typed_ref_new (+1.5 s) and scr_str_alloc (+1.4 s) follow, and the rest is spread in amounts under 1 s. Both large ones scale with the number of records that keep a key list. In the compiler's own build those are the records its 250 minority-order literal sites build (for example IR nodes written kind, key, value, type, loc at 31 sites where the type's table is kind, value, key, optional, type, loc) and the records of observed types that writes fill in a data-dependent order. Two follow-ups address that, neither in this PR: a bit in the record header for "this record keeps a list", so the test is a load and the dispose hook skips every other record, and an order index for explicit literals of a tracked type, so a literal in a minority order keeps no list. The profile says the first alone would bring the time to about the bar, not safely under it.

pi. Plain builds with the Node-run compiler, Intel Xeon 2.50 GHz, each alone, one run per side: base dfa7753d3 2,164 s and 9.27 GB; the branch at 250dafbc98 with the table-choice fix 2,094 s and 10.02 GB (-3.3% time, +8.2% memory). Diagnostics are unchanged: 49 errors at the same 47 sites, none gone, none new. The first revision's pair, against 235d40485: 1,790 s / 8.75 GB and 1,778 s / 8.81 GB for the base, 2,215 s / 9.90 GB and 1,959 s / 10.56 GB for the branch, with 13 more divergence warnings (the index-signature SC6002 sites). The pi build also exposed a cliff nothing else did: choosing a tracked type's table sorted once per key that writes create, and one of pi's keyed-write record types is large enough that the pass did not finish in 27 minutes. It sorts once now, and key-order-witness.test.ts pins it with a 1,500-key type.

🤖 Generated with Claude Code

Activity

15
vercel[bot]6d ago

The latest updates on your projects. Learn more about Vercel for GitHub.

ProjectDeploymentActionsUpdated
scriptc ReadyPreview, v0Nov 7, 2026 4:36pm UTC
vercel[bot]reviewed6d ago
vercel[bot]reviewed5d ago
cramforce5d ago

Converted back to a draft: please do not merge yet. The key orders are right and the round is green, but the natively built compiler compiles itself 7% slower with 7.5% more memory with this branch (three interleaved pairs, numbers in the description), and the bar for this PR is +5% and +6%. Two follow-ups are in progress to get under it: a key list written by a late write names only the keys the record has, and a header bit for "this record keeps a list" so the per-value test is a load.

What this push contains, on top of your merge and review fix (a315e9f256), which are kept as they are:

  • main 50bcfe1b3 merged (no conflicts on top of your merge).
  • The per-value rule described above the fold in the description: a type is tracked only when its own constructions disagree; copies, spreads and clones of a value that keeps a key list hand it on at run time and otherwise cost a test, not a boxed view.
  • Your observation block for the sources of tracked types stays; the equivalent step I had added for tracked sources is removed so there is one mechanism (the one for carried types remains, since those are resolved types your block does not visit). Your key-order-tracked-spread-sources.ts passes unchanged.
  • Keyed writes on closed records, linked copies, and the SC6002 report for index-signature records with declared properties, as described.

Re-run on this head: key-order-witness.test.ts (44), the six key-order corpus programs, the 338-program matrix (287 match), the 320 generated programs (320 match), lint, format, test:ts7 --baselines-only. The last full round (gate, both lanes, both bootstraps, Test262) was on the head before your commit and the last main merge were folded in; a round on this head is running and I will post its result here.

vercel[bot]reviewed5d ago
cramforcereviewed5d ago
cramforce5d ago

Pushed f1c7344a19: three commits on 936c4ce97b and a merge of main b3ff41013c (one conflict, in ir/builtin-signatures.ts, both additions kept).

  • a7aca72d3c: a key list written by a late write names only the keys the record has.
  • 1d0881c5e1, 4cc954f0ce: the review bot's second finding and the same class one step further (a copy of a value that keeps its own key list was given an order index); also no SC6002 for a literal whose type lowers to checked-dynamic objects.

Full round on f1c7344a19 against b3ff41013c: zero-cost gate PASS (32 allowed, 5,282 identical, 118 fail on both sides, no runtime-only row), both native bootstraps, Test262 0 regressed, lanes 22,489 and 19,233 passed with one failure each that passes alone (a known toolchain timing flake; the Effect cluster-entity compile at its 540 s limit on slow machines, 236 s alone).

Still a draft. The natively built compiler compiling itself, 4cc954f0ce against 50bcfe1b3, seven interleaved pairs: memory -1.7% to +1.5%, time +3.9% to +15.2% (mean +10.3%). The bar is +5% time and +6% memory. The numbers, the profile and what is left are in the description.

🤖 Generated with Claude Code

This pull request is a draft

Merging is unavailable until it is marked ready for review.