Problem

Applications embedding camera/video frames already decoded by platform APIs need to render externally owned native GPU resources (for example a D3D11 BGRA SRV or a CoreVideo/Metal surface) without copying pixels through the CPU atlas.

The current published gpui-pre snapshots and renderer APIs do not provide a stable seam for this: downstream applications must maintain a fork of the GPUI snapshot, add a scene primitive, and coordinate resource lifetime until the command buffer completes.

This becomes especially painful when upgrading GPUI Kit: the Kit release pins an exact GPUI snapshot, so a downstream fork based on an older snapshot cannot be combined with the newer gpui-component APIs.

Requested direction

Could GPUI Kit document and/or expose a supported native-surface extension point with:

a platform-neutral scene primitive carrying externally owned resources;
platform-specific import/draw hooks for D3D11 and Metal/CoreVideo;
an explicit completion/lifetime contract;
exact snapshot compatibility guidance for downstream adapters.

I can provide a minimal implementation and tests based on a real camera receiver integration if this direction is acceptable.

Activity

huacnlee1mo ago

Thanks for the detailed write-up. This needs to go to upstream GPUI (zed-industries/zed) rather than into GPUI Kit.

The gpui-pre-* crates are unmodified snapshots of upstream GPUI (for example gpui-pre 0.3.6 = zed@bcf6582), and we deliberately don't carry renderer or behavioral patches in them (see CONTRIBUTING.md). Scene primitives, the D3D11/Metal import and draw paths, and resource lifetime relative to command-buffer completion all live in those crates. GPUI Kit, including gpui-base, has no layer where we could add them.

Upstream already covers part of this: gpui::surface() / PaintSurface draws a CVPixelBuffer on macOS. On Windows, draw_surfaces in the DirectX renderer is currently a no-op, and the wgpu renderer skips surfaces. Making PaintSurface platform-neutral, adding a D3D11 path and an explicit lifetime contract would be a good upstream PR, and your camera-receiver implementation sounds like a strong basis for it. Once it lands in Zed, the weekly snapshot job picks it up, and the next GPUI Kit release that bumps the pin makes it available here.

Until then, the supported way to carry a patch is [patch.crates-io] for the gpui-pre-* crates at the exact version GPUI Kit pins, pointing at your fork. The upstream commit each snapshot was cut from is in the crate description, so you can rebase your fork onto it at each GPUI Kit release.

Haoxincode1mo ago

Thanks — that matches where the renderer work lives. I opened the upstream discussion here:

https://github.com/zed-industries/zed/discussions/64849

Zed does not take blank feature issues, so this is a GPUI discussion rather than an issue. It asks to extend the existing PaintSurface path: a D3D11 BGRA draw on Windows (today draw_surfaces is a no-op), and a Core Video lease held until command-buffer completion on macOS. The wgpu external compositor in zed-industries/zed#60572 is called out as a different seam.

Until a GPUI Kit release bumps the gpui-pre pin onto a snapshot that includes it, I will keep the patch on [patch.crates-io] at the exact snapshot version Kit pins.