#10

Performance: Move VT parsing and text shaping to Web Workers for multi-threaded architecture

Openacc0mplishopened 7mo ago

Summary

Restty currently runs VT parsing (libghostty-vt WASM), text shaping, and WebGPU rendering all on the browser's main thread. Moving compute-heavy operations to Web Workers would significantly improve performance, especially for high-throughput terminal output.

Current Architecture (Single-threaded)

Main Thread (everything)
├── libghostty-vt WASM/SIMD (VT parsing)
├── text-shaper WASM (font atlas, glyph cache)
├── WebGPU rendering
├── DOM/React UI
└── Event handling (keyboard, mouse, IME)

Proposed Architecture (Multi-threaded)

Main Thread              Web Worker 1           Web Worker 2
(UI + Rendering)         (VT Parsing)           (Text Shaping)
    │                        │                       │
    ├── WebGPU render       ├── libghostty-vt       ├── font atlas building
    ├── DOM/React           ├── WASM/SIMD           ├── glyph cache
    └── Event handling      └── PTY data parsing    └── layout calculation

Data Flow

PTY data → Worker 1 (parse VT sequences)
              → SharedArrayBuffer / postMessage → Main Thread (render)
              → Worker 2 (shape new glyphs) → GPUTexture atlas update

Rationale

  1. VT parsing is CPU-bound: libghostty-vt processes raw byte streams through a state machine. During heavy output (e.g., cat large-file.txt, CI logs), this blocks the main thread and causes frame drops.

  2. Text shaping is independent: The text-shaper WASM module builds glyph atlases and computes layout. This work is independent of rendering and can run in parallel.

  3. WebGPU is already async: GPUDevice operations are queued and executed on the GPU. The main thread only needs to submit command buffers, which is lightweight.

  4. SharedArrayBuffer enables zero-copy: With Cross-Origin-Isolation headers, terminal state can be shared between workers without serialization overhead.

Performance Impact

OperationCurrent (Main Thread)With Workers
VT parsingBlocks renderingNon-blocking
Text shapingBlocks renderingNon-blocking
Frame drops during heavy outputCommonEliminated
Input latency during catNoticeableMinimal

Implementation Considerations

  • SharedArrayBuffer requires Cross-Origin-Isolation (COOP/COEP headers) — may need documentation for host apps
  • OffscreenCanvas could allow WebGPU rendering in a worker too (future optimization)
  • Transferable objects (ArrayBuffer.transfer()) can be used for PTY data chunks
  • Fallback to single-threaded mode when Workers are unavailable

Context

We use Restty in KLIC, a Tauri 2.0 desktop terminal multiplexer. With multiple split panes running AI CLI tools (Claude Code, Codex, Gemini) simultaneously, main thread saturation is a real bottleneck. The libghostty-vt WASM core achieves ~90% of native parsing performance, but single-threaded execution limits overall throughput.

Ghostty (native) achieves its performance partly through dedicated I/O and rendering threads. Bringing a similar architecture to Restty via Web Workers would close the gap significantly.

References

Activity

1
wiedymi7mo ago

Make sense, though it will make setup more complex, especially SharedArrayBuffer. I will think about it.