#26

Experimental: render to the HTML DOM on Web instead of a WebGPU canvas

Open1byeopened 5d ago

Status: experiment. The goal is to find out whether a DOM renderer is worth building, not to commit to one. The first deliverable is a spike with a go or no-go report.

Problem

Rake's browser target runs GPUI in Wasm and draws everything to a WebGPU canvas. That gives the same pixels as native, but the browser can't see anything on the page:

  • Accessibility: gpui_web exposes no semantics. docs/future/full-stack-web.md can only offer readable accessibility through hidden prebuilt HTML, and says operable accessibility "is a project of its own".
  • Text: no native text selection, find in page, spellcheck, autofill, password managers or platform IME. Rake has to rebuild text input itself.
  • Reach: WebGPU is still missing or behind flags on some browsers and devices, and a failed start leaves no app.
  • Startup and size: the page waits for Wasm, GPU setup and font loading before the first frame.
  • Web integration: links, right-click menus, browser zoom, printing and extensions all treat the page as one image.

Idea

Keep application state, entities, actions and loaders in Rust and Wasm. Replace only the last step: instead of painting a GPUI scene, build and update real DOM elements. Wasm talks to the page through a small JavaScript layer:

Rust view (view!) -> element tree -> diff -> batched DOM patch ops -> JS applies them
JS event listeners -> event (element ID, kind, payload) -> Wasm -> actions and listeners -> re-render
  • Patch buffer: each render writes create, set-attribute, set-style, set-text, insert, move and remove operations into one buffer. One JS call per frame applies them, rather than a web-sys call per node.
  • Events: one delegated listener per event type on the root. It sends the element's ID and a small payload back to Wasm, which routes it to the same on_click, on_input and action handlers as native.
  • JS calls for features: platform features without a Rust binding (clipboard, file pickers, notifications, IntersectionObserver and so on) go through the same typed boundary as rake::web::run_worker: JSON or small typed messages, no Rust closures held by JS.

Where the renderer sits

OptionHowVerdict
A. view! tree to DOMview! lowers to a renderer-neutral element tree. A DOM backend diffs and patches it.Proposed for the spike
B. GPUI element tree to DOMInspect GPUI's built Divs and their StyleRefinement after render and map them to DOMWorth testing alongside A. Depends on GPUI internals
C. GPUI scene to DOMTurn painted quads and glyphs into absolutely positioned elementsRejected: loses semantics, text and layout, which are the point

docs/future/full-stack-web.md rejected generating HTML straight from view! because "it would be a second renderer that ignores runtime state and component output". Option A is a second renderer, so the experiment has to answer that directly:

  • It runs live, with runtime state, so it doesn't share the static-HTML problem of drifting from the app.
  • Component output is the real gap. Components written with raw GPUI builders, custom Element impls, canvas() and paths have no DOM form. The spike must measure how much of the examples that is.
  • Fallback for unsupported elements: a canvas island, a <canvas> sized by the DOM layout that runs the existing GPUI renderer for that subtree only, if the spike shows it's practical. Otherwise a build-time error naming the element.

Styling

Use one style model for both renderers, so classes and style={|s| ...} closures keep working:

  • Class utilities already compile to GPUI style calls. The DOM backend applies them to a GPUI StyleRefinement and converts that to CSS.
  • GPUI layout is flexbox and grid through Taffy, which maps closely to CSS. Record every difference found (defaults, min-size, overflow, text wrapping, rounding) in a parity table.
  • Theme tokens become CSS custom properties, so theme changes update one stylesheet rather than every element.
  • Fonts come from CSS @font-face. Pixel parity with native is a non-goal. Close visual parity is the goal.

What stays Rust-side

  • GPUI's App, entities, cx.notify(), actions and key bindings, if App can run on Web without a GPUI window. This is the spike's first question. If it can't, the experiment needs a reduced runtime and should say so.
  • Focus: DOM focus is the source of truth, mirrored into GPUI's focus handles so on_action and key contexts still work.
  • Text input: the DOM backend renders <input> and <textarea> natively and reports edits as the same events as rake::ui::input (see the gpui-base migration plan). IME, selection and autofill then come from the browser.

Effects on the rest of Rake

  • Accessibility: ARIA attributes from view! (aria-label, roles, tabindex) become real attributes, which makes operable accessibility on Web nearly free.
  • Full-stack Web: semantic HTML from layouts and the hidden fallback in docs/future/full-stack-web.md could become real server-side rendering with hydration. Record whether the DOM backend changes that plan; don't change it yet.
  • Inspection and agents: inspection maps to real DOM nodes, so CDP queries, screenshots and rake ctl should mostly use the browser's own tooling. Element IDs must stay the same between native and DOM.
  • Motion and lists: motion frames map to CSS transforms and opacity, and <list> (#24) maps to a DOM virtual list. Neither should block the spike.
  • Bundle choice: the renderer is a build option, for example [package.metadata.rake.bundle.web] renderer = "dom", with the WebGPU canvas staying the default until the experiment ends.

Spike (phase 0)

Build the smallest version that answers the questions below, behind a feature flag, on one example (examples/view or the counter in the README).

  1. Does GPUI's App, with entities and actions, run in Wasm without a window, on the current pin and on gpui-pre?
  2. What share of the elements in examples/view, examples/notes and examples/inspector can render as DOM with no canvas island? List the rest by kind.
  3. Can a canvas island share a page with DOM content: sizing, input routing, focus and device pixel ratio?
  4. Cost of the patch buffer against direct web-sys calls, for a 1,000-row update and for typing in an input.
  5. Startup time to first useful paint, and bundle size, against the WebGPU build of the same example.
  6. Layout parity: screenshots of the same view on native and DOM, with differences listed.

Write the results to .agents/reports/web-dom-renderer-spike.md with a recommendation: continue, continue with a smaller scope (for example DOM only for routes built entirely with view!), or stop.

Non-goals

  • Replacing the WebGPU renderer. Both stay until the experiment ends.
  • A DOM renderer on native platforms or webviews.
  • Running arbitrary GPUI crates, such as GPUI Kit components, as DOM. They may run as canvas islands or not at all.
  • Pixel-identical output with native.
  • Server-side rendering and hydration. Note the opportunity, but leave it for later.

Open questions

  1. Should the element tree lower from view! only, or should Rake components also get a DOM form so rake::ui works on both?
  2. Who owns the JS layer: generated by rake bundle, or a small checked-in runtime like scripts/embedded-devtools.mjs?
  3. How do GPUI key bindings coexist with browser shortcuts and text fields?
  4. What happens to Rake's overlays (popovers, menus, tooltips)? The browser's top layer and popover may handle much of it.

Done when

  • The spike runs one example as DOM behind a flag, in Chrome, Firefox and Safari.
  • The report answers all six spike questions with numbers and lists element kinds that need canvas islands.
  • A clear recommendation is recorded, and this issue is updated with it or closed.

Activity

0

No activity yet.