#2115

Firefox WebDriver BiDi attach support (connect works at network level, protocol is Chrome-CDP only)

Opendmccarty666opened 15d ago

Firefox (WebDriver BiDi) attach support — connect currently Chrome-CDP only

Context

We run a self-hosted OpenMausBot server (which bundles agent-browser 0.37.0 as its browser-automation backend) with per-bot Cua XFCE Docker VMs (trycua/xfce-cua images). The VM ships Firefox ESR as its desktop browser — that's the browser the bots and humans actually see/use inside the VM desktop (TigerVNC + noVNC).

When a bot attaches agent-browser to that Firefox (so the "browser" lives in the VM instead of a second headless browser), the attach works at the network level — Firefox's remote agent is listening — but agent-browser can't speak the protocol.

Repro

Firefox ESR 128 (Debian package, firefox-esr), launched with:

firefox-esr --no-remote -profile <pf> --remote-debugging-port 53645
# prefs.js:
#   devtools.debugger.remote-enabled = true
#   devtools.debugger.server.ip = "0.0.0.0"
#   devtools.debugger.server.port = 53645

Listening and healthy:

WebDriver BiDi listening on ws://127.0.0.1:53645

Relayed to the host at 127.0.0.1:9222 (via docker bridge IP, trivial TCP proxy). Then:

$ agent-browser connect http://127.0.0.1:9222 --json
{"success":false,"error":"All CDP discovery methods failed for 127.0.0.1:9222:
  /json/version: Failed to connect ... error sending request for url (http://127.0.0.1:9222/json/version);
  /json/list: Failed to connect ...;
  WebSocket: WebSocket connect failed at ws://127.0.0.1:9222/devtools/..."}

The CDP server / discovery paths Firefox's remote agent implements are different (/ returns the BiDi info page; the session is a BiDi WebSocket with webDriverBiDi handshake, no /json/*). Auto-launch fallback then fails too (Chrome exited early ... DevToolsActivePort) — expected, it's Firefox not Chrome.

Request

Support for WebDriver BiDi endpoints in connect (or a --protocol bidi flag), so agent_browser_*-style tools can attach to Firefox. Firefox's BiDi surface covers the same primitives the CLI already exposes: browsingContext.navigate/create/close, script.evaluate, input.performActions, browsingContext.captureScreenshot, DOM mutation via script, etc. A dial-detect (probe /json/version vs /) picking the dialect automatically would be ideal.

This would make agent-browser work against the default browser of Linux VM/sandbox images (Cua, and likely others) instead of forcing a Chrome install.

Env

  • agent-browser 0.37.0 (Linux x86_64, Ubuntu 24.04 host; same inside Debian VM)
  • Firefox ESR 128.x (firefox-esr Debian trixie-security), profile prefs above
  • Host: 7.0.0-34-generic, container: trycua/xfce-cua (driver 0.20.0)

Happy to test a pre-release/branch against this exact Firefox-VM setup.

Activity

0

No activity yet.