Scene API north star: tree orchestration + program protocol

Status

Design approved as the north star for the scene API; implementation is partial on branch feat/scene-view (commits below, not yet pushed). This issue is the single entry point for all context on the redesign — point any future work (human or agent) here first.

History

• #185 — Scene tree phase 1 + Mesh→Geometry rename (merged).
• #211 — free functions / minimal kernel (merged) — retained-state ownership model this issue builds on.
• #205 — superseded by this issue. Its createSceneView/SceneView MVP contract was pivoted mid-implementation once the boundary below was fixed: material is gone (→ program), and the public entry point is a function (renderTree) instead of a constructed view object. #205's acceptance criteria and edge cases (ownership, fingerprint sharing, opacity re-keying, lifecycle) are carried forward here and into the sub-issues, reinterpreted under the new vocabulary.
• #212 / #213 — WGSL reflector fix for bare package specifiers as binding/struct-member types. Required for file-first authoring (import { SceneCamera, SceneModel } from "@vgpu/wgsl-std/scene" used directly as a binding type). #213 is mergeable, CI green.
• #207 — the old 0.2.0 parent; this issue is now where scene-specific work is tracked (see the comment on #207 linking here).
• #206 — compilation orchestration follow-up; still a separate, existing issue, but its target surface is now renderTree/sceneView vocabulary (see the comment on #206 linking here).

The boundary (the user's governing rule)

The design targets functional programming (factories + composition), explicitly not Object3D-style inheritance. A user's custom lighting/shadow/shader system must never have to fight anything predefined by the scene package.

The tree

import { group, mesh, perspectiveCamera } from "vgpu/scene";

const world = group();
const character = group({ label: "hero", position: [0, 1, 0] });
character.add(mesh(capsule(), skinProgram), mesh(box({ size: 0.3 }), visorProgram));
world.add(character);
character.rotation = [0, Math.PI / 2, 0]; // TRS → lazy worldMatrix
const cam = perspectiveCamera({ fov: 60, position: [0, 2, 5] });
world.add(cam); // a camera is a node (gets parenting for free)
world.find("hero");

// a custom "Mesh" is composition, never a subclass:
function fireball(r: number) { return mesh(sphere({ radius: r }), fireProgram); }

program (rename approved: there is no "material" at the scene level)

A program is shader + pipeline state + bindings. Two overloads; SCENE_PROGRAM (Symbol.for("vgpu.scene.program")) is the low-level escape hatch for class-based/third-party implementers. Full contract, ABI, and renames: see #214 (program protocol sub-issue).

// STATIC (90%):
const rockProgram = program(rockShader, { cull: "back" });
rockProgram.set({ roughness: params }); // set() is the house verb for wiring buffers

// DYNAMIC (callback per geometry shape — deferred compilation):
const skinProgram = program<SkinAttributes>((attributes, ctx) => ({
  shader: skinnedShader,                  // ctx: { gpu, node, use() }
  draw: { cull: "back" },
  bindings: { bones: bonesOf(ctx.node) }, // ctx.node = per-mesh state channel
}));

Callback contract: runs once per (program × geometry shape), never per frame; memoized by a fingerprint the view derives (hash of final WGSL + draw state + queue + optional variant key — never author-declared: sub-keying is a silent, incorrect pipeline share). Returns a description (declarative); the view compiles it (sync at first use; the PipelineAcquisition choke point stays intact for future orchestration). Must be sync and pure.

Root rule: whoever declares a binding, supplies it. group(0) belongs to the view (camera+model, connected by reflection — the view recognizes SceneCamera/SceneModel by layout regardless of naming; anything unrecognized in group(0) is a clear creation-time error). group(1)+ belongs to the author. The core field from earlier drafts is eliminated (reflection deduces it). File-first authoring enabled by the #212/#213 fix: import { SceneCamera, SceneModel, SceneVarying } from "@vgpu/wgsl-std/scene" works directly as a binding type. WGSL paths: { fragment } (view prepends the prelude: group(0) + SceneVarying + vertex std — the contract can't be broken) or { shader } (full module, custom vertex: instancing/skinning).

ABI (frozen, bytes verified naga/Dawn, on feat/scene-view with a pending split amendment)

• group(0) binding 0 = SceneCamera { viewProjection: mat4x4f, position: vec4f } (80 B); binding 1 = SceneModel { matrix: mat4x4f, normalMatrix: mat4x4f } (128 B). normalMatrix = CPU inverse-transpose (correct Lambert under non-uniform scale).
• Lights leave group(0): SceneLights (count-prefixed, capacity 8, 672 B) + vgpuSceneLighting move to the vgpu/shading pack as its own @group(1). @vgpu/wgsl-std/scene splits into core (camera/model) and scene/lighting.
• Vertex contract: location 0=position, 1=normal (optional), 2=uv (optional), custom 3+; location 8-11 reserved for instanceMatrix (vertex-std convention).

Render: a function, not a class

renderTree(gpu, world, cam, { target: screen, frame? }); // composable with `frame`; standalone without it

Retained state (per-MeshNode records, pipelines, model buffers) lives in the kernel, keyed by root (services + ownership from #211); gpu.dispose() releases it. An explicit sceneView() handle (compile()/.stats/#206) is a deliberate future addition, not MVP. Render runs in four phases: walk (DFS with visibility pruning + hooks + accumulates visibleNodes) → systems beginFrame({ visible, camera, frame }) → bindings (draw.set camera/model/program) → pass opaque→transparent in DFS order (no sorting). Full spec: #215.

Lights / systems (optional vgpu/shading pack)

Light nodes stay in vgpu/scene (pure data, still get parenting); their interpretation lives in the pack via SceneSystem (SCENE_SYSTEM symbol): ctx.use(system) auto-registers; beginFrame receives the visible nodes from the walk (zero double-traversal, pruning for free); disposed together with the retained state. VGPU-SCENE-LIGHTS-CAPACITY moves to the pack. Core keeps normalMaterial (mesh()'s default, ~150 B) and shaderMaterial/raw program. CI fixtures: scene-renderer (batteries-included, closes the pending #204 acceptance item) + scene-view-custom-material (structural exclusion of the pack — mechanical gate for the tree-shaking promise). Shadows/multipass are explicitly out of the MVP — a future additive SceneSystem capability, not something program will ever own. Full spec: #216.

Instancing + hooks (API reserved now; implementation is a fast-follow)

const forest = instancedMesh(pine(), barkProgram, {
  count: 500,
  instance: { tint: "float32x3", sway: "float32" }, // schema → typed writer + locations 12+
});
forest.instances.set(17, { matrix: empty.worldMatrix, sway: 0.5 }); // partial OK; CPU staging + dirty-range, 1 upload/frame
forest.count = 350; // shrink without reallocating

// onBeforeRender: fires on ANY node. Timing contract: fires during the walk, DFS order, at the
// node's own visit, with its worldMatrix + ancestors already resolved; reading any node's
// worldMatrix is valid (lazy); writing GPU data takes effect this frame; writing transforms of
// already-visited nodes takes effect next frame; nodes pruned by visible never fire.
forest.onBeforeRender = () => spawnPoints.children.forEach((empty, i) =>
  forest.instances.set(i, { matrix: empty.worldMatrix }));

// path 1 (pack-friendly): instanceMatrix at locations 8-11, vertex std auto-detects and multiplies.
// path 2 (custom): storage buffer + @builtin(instance_index) via { shader } — GPU-driven / compute.

Batching: level 1 free today (shared fingerprint = one pipeline); level 2 = mergeGeometries util (pre-process, orthogonal); level 3 dynamic/indirect/culling = phase 3 (low-level indirect already exists). Future sugar: forest.instances.follow(nodes). Full spec: #217.

What already exists on feat/scene-view (unpushed commits, base main@1905a0c)

Renames pending on implementation

material → program in: mesh(geo, program), MeshNode.program, SCENE_MATERIAL→SCENE_PROGRAM, SceneMaterialContext→SceneProgramContext, etc. createSceneView → internal only; the public entry point is renderTree. The abstract SceneMaterial class degrades to an interface (any object carrying the symbol qualifies). Exception: normalMaterial/shaderMaterial built-in names are kept as-is on purpose (see #214).

Sub-issues

• #214 — Scene: program() factory, SCENE_PROGRAM protocol and core ABI split
• #215 — Scene: renderTree() with kernel-owned retained state
• #216 — Scene: vgpu/shading pack — built-in programs and the lighting SceneSystem
• #217 — Scene: instancedMesh with typed instance schema and onBeforeRender
• #218 — Scene: follow-ups backlog

Related, existing, not a sub-issue of this one: #206 (compilation orchestration for renderTree/sceneView).