What
scriptc build|run … --node-island-decision=<file> keeps the Node island decision a build reaches in a file, and a later build of the same entry starts from it.
A build decides its islands in rounds: it lowers the whole program, islands the packages and modules whose lowering refused, loads again with their declarations and lowers again, until a round adds nothing. Only the last round's result is the build's. On a large program the earlier rounds are most of the build, and between two builds of the same program their answer rarely changes.
The file holds the islands the last round started from: the island packages with their class and first reason, and the island modules. Those are already parameters of prepareExecutableModule (lowerIslands, lowerModules), so a build that finds the file starts its first round where the recording build's last round started. Nothing else about the round changes:
- The first decision (refused, marked and patched packages) is made from the load as always.
- The lowering's own refusals and the fence rules judge the program as it is now, so a program that needs another island (a new import that refuses, a new dependency with a fence that surely fires) gets it by one more round, and the file then holds the grown set.
- An entry whose package directory or module file is gone, whose version changed, or that the policy now keeps native or unloaded is dropped and decided again. A file recorded for another entry, island policy,
--npm-staticrequest or target platform is not replayed. - An island the program no longer needs stays until the file is deleted. Finding that out would take the native lowering the file saves; the package keeps running in Node, correctly.
Without the option nothing is read or written and no round changes.
Why
On a large multi-package program (three island rounds, 14 island packages) the build lowers 6,975, then 5,440, then 4,790 reachable functions to report the third round's result. Measured in 32 vCPU sandboxes, one build at a time, each pair in one sandbox:
| build | CPU | wall | peak RSS | rounds | diagnostics |
|---|---|---|---|---|---|
| this head, option given, no file yet (decides from scratch and records) | Xeon 2.50 GHz | 1,296.00 s | 7.64 GB | 3 | 66 error lines |
| this head, replay | Xeon 2.50 GHz | 426.11 s | 4.38 GB | 1 | byte-identical |
main 714482d49, no option | Xeon 2.90 GHz | 914.30 s | 7.50 GB | 3 | byte-identical |
| first commit, records | Xeon 2.90 GHz | 903.67 s | 8.01 GB | 3 | byte-identical |
| first commit, replay | Xeon 2.90 GHz | 251.97 s | 5.00 GB | 1 | byte-identical |
| first commit, no option | Xeon 2.90 GHz | 853.05 s | 7.78 GB | 3 | byte-identical |
| first commit, replay | Xeon 2.90 GHz | 254.56 s | 4.54 GB | 1 | byte-identical |
A replayed build is 3.0 to 3.6 times faster than the build that decides from scratch on the same machine, and reports the same 544 lines. The recorded file holds the 13 packages the rounds grew, and every replay left it byte for byte as the recording build wrote it.
The measurements behind the choice of this mechanism over a decision made before lowering (the set depends on refusals in function bodies and on whether the whole program fails; a marker-based predictor would move packages that compile natively today into Node in 49 corpus programs) are in the design note this PR implements the first slice of.
How
node-islands/decision-file.ts: the file's format, its key, the read (with per-entry validation) and the write (same decision, same bytes; written through a rename).executable/prepare.ts:prepareExecutableBuildreads the file, callsprepareExecutableModulewith the recorded state, and writes back the state of the round whose result it returns. Each round that returns a result of its own reports the state it started from (onDecided); a round that retries leaves that to the retry.index.ts,native/prepare.ts: both compile paths callprepareExecutableBuild. The executable cache key and the native frontend cache key take the file's digest; the native executable cache is bypassed with the option, as it is for a profile.cli/usage.ts,cli/command.ts,packages/cli/src/bootstrap.ts: the option, forbuildandrun.
This touches the round entry point that #2881 restructures. The two compose: in #2881's loop the state to record is its request.
Proof
- A replay ends where a build from scratch ends (
packages/compiler/test/island-decision-replay.test.ts): for a fence-report island, a lowering-refusal island and a sure-fence island, the build from scratch takes two rounds and the replayed build one, with the same serialized IR, warnings and island list; a failing build's replay reports the same diagnostics from one round; a record that misses an island is grown by a round and written back; a dependency added after the record gets the island a build from scratch gives it; a recorded package whose version changed is decided again; a program without islands records an empty decision and builds as before. - Command line (
packages/cli/test/node-island-decision.test.ts): the same LLVM text and the same SC8001 notes from scratch, while recording and on replay, with onefrontend-loadinstead of two; an executable is cached by the file's bytes;coveragerefuses the option. - File (
node-islands/decision-file.test.ts): round trip, relative paths, key mismatches, dropped entries, unreadable files, the digest. - Zero cost: the zero-cost gate over the whole corpus (
--gate-corpus=all), this head0d30b3d11against its base7954cf5d4, passes: 5,178 programs identical (IR, LLVM and release executables), 96 fail on both sides, 0 differ, 0 runtime-only. Compile time within tolerance: benchmarks +0.01 % [95 % CI −0.77 %, +0.94 %], the compiler compiling itself +0.76 % [−0.09 %, +1.04 %], fixtures −0.00 % [−0.29 %, +0.28 %]. - Native bootstraps:
tests/harness/self-hosting-native-driver.test.tsandtests/harness/self-hosting-llvm-emitter.test.tspass on this head (3 tests), and pass underSCRIPTC_SAN=1on the first commit's diff (3 tests); the second commit removes one recorded field. pnpm lint,pnpm format:check: clean.
🤖 Generated with Claude Code
Activity
14The red macOS job is not from this change. tests/corpus/6125-constants-module.cjs fails there on one line: natively constants.hasOwnProperty("O_SYMLINK") === ("O_SYMLINK" in constants) prints false (the constant exists on macOS only). That program is byte-identical in IR, LLVM and executable between this branch and its base in the zero-cost gate, and nothing here is platform-specific. The macOS jobs are skipped on pushes to main and on most pull requests, and this one runs them because it touches the command line and cache paths, so it is the first to show it. The likely source is #2996 (answers for in and hasOwnProperty on builtin module namespaces from the member census). In the first run tests/corpus/8830-process-report-island also failed once on the macOS arm64 artifact job.
This PR and #2881 both change the island rounds in executable/prepare.ts and conflict there (nine hunks).
If #2881 lands first, this one needs the port on branch feat/node-island-decision-wp1 (head af0593c16): #2881's head with this PR merged in. In #2881's loop the state to record is the request of the round whose result the loop returns, or, for a build that keeps its own result after its fence attempts failed, the islands that build started from. The decision tests there count a round by its lowering, because #2881's rounds share the load of the program without islands. The port's own tests pass (211 across the decision, round and CLI suites); I will merge it into this branch once #2881 is on main.
If this PR lands first, #2881 takes the same resolution.
Follow-ups to this PR (dropping islands the program no longer imports, re-deciding fence islands when the build's outcome changes, a check mode, a project key, coverage replay) are in progress on top of that port.
Merged main (97d372266) into this branch: 54b52bcc4.
What conflicted. index.ts and native/prepare.ts, both against #3218 (lowered programs), which wraps the frontend of both compile paths in prepareLoweredProgram. The frontend it wraps is this branch's prepareExecutableBuild, so:
- a build that writes a lowered program (
--emit-lowered) still reads its island decision and records it; - a build from a lowered program (
--from-lowered) runs no frontend, so it neither reads nor writes the decision file, and it is refused by name when the decision option differs from the build that wrote the file (does not match this build's options: … option nodeIslands).
The executable cache key keeps main's tier field beside the decision digest.
What merged without a conflict and needed a line. Main's #3260 requires a cache key rule for every build and run option (executable-cache.test.ts). --node-island-decision gets one: the island key holds the file's path and the digest of its bytes.
Re-run on the merged head (Linux x64 Sandboxes, Intel Xeon 2.50 GHz):
- Zero-cost gate,
--gate-corpus=all, against main97d372266: PASS, 5,295 identical, 108 failing on both sides, none differ, no runtime-only row. Compile time: benchmarks −0.13% [−1.13%, +0.87%], the compiler's own build −0.80% [−1.94%, −0.25%], fixtures −0.34% [−0.84%, +0.16%]. - This branch's tests (
decision-file.test.ts,island-decision-replay.test.ts,packages/cli/test/node-island-decision.test.ts), theexecutable,nativeandcliunit directories andtests/harness/lowered-program.test.ts: 217 passed, 1 failed; the one isexecutable-cache.test.ts"every build and run option…", which listed only main's three lowered-program options (main was red there; #3317 has fixed it since). With main's fix applied to a copy, that file passes, 33 of 33, this option's rule included. All ofpackages/cli/testandpackages/compiler/src/cli: 136 passed. - A recorded decision together with a lowered program, on
tests/corpus/6829-node-island-declared-types(tools/3117-probe.shin the fleet notes): recording with--emit-loweredtakes two frontend loads and writes both files; replay takes one;--from-loweredwith the decision option builds from either lowered file with no frontend load; every build's LLVM text is byte-identical to the build from scratch. - Island corpus programs, differential and LLVM-differential: 228 passed.
- Plain native bootstrap (
self-hosting-native-driver.test.ts): passed. pnpm lint,pnpm format:check,pnpm test:ts7 --baselines-only: clean.
Not run on this head: the sanitized bootstrap, the sanitized island corpus and the full lanes.
Main has moved since (8411368da); this head still merges with it without a conflict.
Correction to my comment above: this head merges with current main without a textual conflict, but the merge does not build, as this PR's CI shows (src/executable/prepare.ts(115,18): error TS2554: Expected 3 arguments, but got 2, and the same in decision-file.test.ts(66,10)).
Main's #3297 gave nodeIslandPolicy a third parameter, the entry module. This branch calls it with two arguments in prepareExecutableBuild (executable/prepare.ts) and in the test helper policyOf (node-islands/decision-file.test.ts); both need the entry path, as main's other calls pass it. Everything in the comment above was measured on 54b52bcc4, which is this branch merged with main 97d372266, before #3297.
I am not pushing the two-line fix here: the island-decision stack is landing through the startup-2 train, and the fix is written up for its conductor.
The latest updates on your projects. Learn more about Vercel for GitHub.