Build Node island programs with the native compiler

What

The natively compiled compiler (the one the scriptc package ships) now builds programs that need a Node island. Until now it built them into executables that could not start.

The defect

For a program with a Node island, the native compiler printed the SC8001 note, exited 0 and wrote an executable that exits 1 at start:

$ scriptc build main.ts -o prog
/vercel/sandbox/nci-repro/mini/main.ts:2:1 - note SC8001: package 'tiny@1.0.0' runs in a Node island: marked: marked "node" in scriptc.packages

  hint: it runs in Node.js loaded into the program on first use; --node-islands=off makes this an error
/tmp/nci-out/native-main/mini/prog
$ ./prog
scriptc: this program's Node island module manifest is malformed

(Output of main 549aa83d6's native compiler, trimmed: the note's code frame and the deprecation line of the --emit-ir the run also passed; the command is shown without its directories. main.ts is import { greet } from "tiny"; console.log(greet("island")); with "scriptc": {"packages": {"tiny": "node"}}.)

The Node-hosted compiler builds the same program correctly against the same libnode pack (bridge ABI 10, Node 24.22.1), so the pack is not the cause. prepareExecutableModule is shared by both drivers and decides and lowers the islands. Three steps that follow it existed only in index.ts, and each shows when the two compilers' outputs are compared:

The last row is the message: the manifest lists files and the executable carries none. It reproduced for every way an island comes about (a package marked "node", a package whose lowering is refused, the pre-scan, the fences, a module of the program, the @scriptc/node call API). requireModule() failed differently: the native compiler had no island module resolver, so the load became a runtime fence and the executable ended with Uncaught Error: 'requireModule (@scriptc/node) of './plugin.mjs', which does not resolve from …. --libnode was accepted and ignored. No test built an island program with the native compiler.

The change

Two commits. The first refuses such programs by name in the native driver (SC3002) and stands alone, in case the second needs more review. The second replaces the refusal with the build.

• One copy of the island steps. executable/islands.ts finishNodeIslands holds the three steps. index.ts calls it and loses its copy (57 lines out, 15 in); native/prepare.ts calls it after the shared prepare and passes the module bytes to the emitter. Nothing was copied into the native driver.
• An island build host (node-islands/host.ts, replacing resolve.ts) answers what depends on where the compiler runs: module resolution for island code, the libnode pack installed with the compiler, the runtime sources the deferral probe host is built from, and the static effect gate. The Node-hosted compiler installs Node's own resolution (host-node.ts); the native compiler uses the shared filesystem resolver, as a compiled program's require.resolve does, and its toolchain manifest (the distribution already ships runtime-sources). The gate parses with the TypeScript 5 parser, which the native compiler does not have; it only words the deferral trace (the probe decides), so the native trace says "recorded load is deterministic" without the gate's remark.
• The probe walks typed IR. snapshot.ts found island loads by reflecting over the module's objects; it now uses everyStmtList, which the native compiler compiles and the Node-hosted one runs the same way. libnodeRecordFor / libnodeCacheKeyFor take the native target from the driver, so the native compiler never reaches the host inspection behind libnodeTarget (process.report, which it does not compile).
• Native caches. A frontend cache entry of a program that calls into Node keeps the libnode pack's identity and is reused only while it is unchanged; a --libnode=beside build always lowers, since it places the pack and the modules from the lowered module; island builds record no unchanged build and take no token-equivalent reuse (their LLVM carries the island modules' bytes). A program without islands takes none of these paths: finishNodeIslands returns at once when the module has no island record.
• Tests. The bootstrap contract's command phase builds one island program per kind with the native command (6870, 8810, 6817, 6819, 8424, 6850, and a package named with --node-island=), runs each without Node on PATH against the program under Node, and compares its LLVM (bytes) and IR (as a value) with the Node-hosted compiler's build of the same command line. CI's bootstrap command phase gets the libnode pack. native/prepare.test.ts covers the module bytes, the eager form without a pack, SC8005 without a pack, and the cache rules.

Proof

Sandboxes are 32 vCPU / 64 GB Xeon 2.50GHz or 2.90GHz, named per line; the libnode pack is the candidate's (bridge ABI 10, Node 24.22.1). Runs marked "pre-rebase" were on the same two commits over main a09254dd3. "This head" in the tables is 67234eba7, the two commits over c3e73cbfd. The head pushed is the same two commits rebased once more, without conflict, over 716cb8626 (17 commits later): on it the build, lint, format check and the 381 focused tests were rerun and pass; the bootstrap contract and the gate were not rerun there.

The native compiler against Node, every corpus program that needs a libnode pack (95 programs with // @requires-libnode; native compiler from pnpm build:native-compiler, 955 s; 2.50GHz; pre-rebase):

The program that differs from Node under both compilers is 6847, by the process id in Node's deprecation warning, which this plain runner does not normalise. The two whose LLVM and IR differ are 6825 and 6845: see "Not in this PR". Main's native compiler was tried on nine of these programs; none of its executables started.

The Node-hosted compiler, base against candidate, on the same 95 programs (the zero-cost gate has no libnode pack, so island programs fail to build on both of its sides; this is the comparison it cannot make; 2.50GHz; pre-rebase): 95 of 95 build with both, LLVM byte-identical 95, IR byte-identical 95, diagnostics identical 95. Moving the steps into one function changed nothing the Node-hosted compiler writes.

The island steps compiled natively against the same steps under Node (a scratch program around finishNodeIslands, fed the un-finalized IR of main's native compiler; five programs × exe, ir, llvm; the port over 549aa83d6): 15 of 15 give the same libnode record, module bytes (length and SHA-256) and load calls with their full snapshot text, including the eager decision for the module with effects (6842).

Bootstrap contracts (seed built as CI builds it, then pnpm vitest run tests/harness/self-hosting-native-driver.test.ts, SCRIPTC_LIBNODE set):

The two full runs used the contract before its island block was a named step; the command-phase runs use the contract as committed and show bootstrap: Node island programs match Node and the Node-hosted compiler took …s. The rebuild phases were not rerun on this head.

Zero-cost gate, --gate-corpus=all, this head 67234eba7 against c3e73cbfd: PASS. 5,241 programs identical, 102 fail to build on both sides (all 102 are SC8005, island programs without a pack), none differ, no runtime-only row. Compile time: bench +0.25% [95% CI −0.83%, +1.53%], self −0.05% [−1.61%, +0.57%], fixtures −0.44% [−1.86%, +0.28%], all within tolerance (five pairs per fixture). Pre-rebase, against a09254dd3: PASS, 5,214 identical, 97 both-fail, none differ.

This head also: pnpm lint and pnpm format:check clean; 381 tests in 28 files pass (packages/compiler/src/node-islands, native, executable, and tests/harness/node-island{,-deferral,-handles,-order}.test.ts with the pack).

--libnode (the port over 549aa83d6): --libnode=/nonexistent/pack on an island program is SC8005 from both compilers (main's native compiler exited 0). --libnode=beside writes the same <out>.libnode/ (pack and modules) from both; the moved output runs without SCRIPTC_LIBNODE and matches Node; a native rebuild after deleting the directory places it again.

The first commit alone (the refusal, over 549aa83d6): unit test; plain bootstrap pass (seed 1,104 s, contract 1,539 s, 2.90GHz) after a first attempt whose seed build ran into the contract's 30-minute limit on a sandbox busy with other jobs; sanitized bootstrap pass (4,026 s); gate PASS on a second round (5,195 identical, 96 both-fail, none differ; fixtures −0.13% [−1.25%, +1.34%], five pairs) after a first round that reported fixtures +2.58% [+2.13%, +3.04%] from two pairs per fixture with identity clean.

Not in this PR

• The native compiler puts more packages in islands than the Node-hosted one. It refuses a package's require() of another package (SC1010: the 'pb' module is not supported yet) where the Node-hosted compiler compiles both natively. With this PR such a program builds and matches Node, with the requiring package in an island; with --node-islands=off it is refused by name. That is why 6825 and 6845 are the two corpus programs whose LLVM differs between the two compilers. It predates this change (main's native compiler makes the same decisions) and belongs to the native program loader.
• The two compilers serialize a module's top-level members in different orders (globals before or after records and unions), for any program, so their IR is equal as a value and not byte-identical. The contract compares it the way it compares the compiler's own IR.

🤖 Generated with Claude Code