#2208

feat(firefox): dashboard live view, bidi module and docs

Openlahfirwants to merge intofrom
  1. Firefox sessions did not show in the dashboard live view, and the engine had no documentation.
  2. This adds the dashboard live view for Firefox, moves the engine into a bidi protocol module, and documents it across the docs, README, help and skill.
  3. Builds on #2207.

Demo

The terminal drives a Firefox session over WebDriver BiDi; the right side shows that Firefox page as each command runs.

https://github.com/user-attachments/assets/a0166b7b-9f59-4ac2-9511-c7185b33e7da

Architecture

flowchart TB
  subgraph Clients
    CLI[CLI]
    MCP[MCP server]
    DASH[Dashboard]
  end

  CLI & MCP & DASH -->|JSON over daemon socket| EX

  subgraph Daemon["Daemon (native/actions.rs)"]
    EX[execute_command]
    GATES[Policy + confirmation gates<br/>implicit launch / relaunch]
    ROUTE{engine == firefox?}
    CDPH[Existing CDP handlers<br/>Chrome, Lightpanda, Obscura,<br/>--cdp, providers]
    WD[WebDriver path<br/>Safari / iOS]
    EX --> GATES --> ROUTE
    ROUTE -->|no| CDPH
    ROUTE -->|no, provider ios/safari| WD
  end

  ROUTE -->|yes| BA

  subgraph Bidi["native/bidi/ (BiDi protocol module)"]
    BA[actions.rs<br/>SUPPORTED / CHROME_ONLY lists<br/>dialog guard, dispatch]
    subgraph Handlers["Handlers (any BiDi browser)"]
      PAGE[page: navigate, history, waits, content]
      TABS[tabs: t1..tN ids, new/switch/close]
      FORMS[forms: fill, type, press, select, check, upload, inserttext]
      INTER[interact: click, hover, drag, scroll, tap, dialog]
      QUERY[query: get/is, find/getby*, frame]
      CAP[capture: screenshot, annotate, pdf]
      STOR[storage: cookies, storage, state save/load]
      NET[network: route, HAR, requests, headers, auth, downloads]
      EMU[emulation: viewport, device, geo, media, offline]
      SNAP[snapshot: tree + refs]
      TGT[target: ref / selector resolver]
      INP[input: key + pointer action builders]
      STRM[stream: dashboard polling + input]
    end
    SESS[session.rs FirefoxSession<br/>active/top context, tab ids,<br/>ref side table i64 to sharedId,<br/>event pump]
    CLIENT[client.rs BiDi WebSocket client<br/>send, send_typed, events]
    TYPES[types.rs LocalValue, RemoteValue,<br/>BytesValue, Cookie, events]
    FF[firefox.rs find, validate, launch<br/>temp profile, prefs, process group]
    DEVT[firefox_devtools.rs RDP client<br/>accessibility tree, media simulation]
    BA --> Handlers
    Handlers --> SESS
    SESS --> CLIENT
    CLIENT --- TYPES
    SESS --> DEVT
    FF --> SESS
  end

  subgraph Shared["Shared with Chrome (reused, not copied)"]
    JS[Page JS constants<br/>element.rs / interaction.rs]
    RENDER[Snapshot renderer + RefMap<br/>build_tree / render_tree]
    POLICY[DomainFilter, route matching,<br/>state format, screenshot/pdf/upload helpers]
  end

  FORMS & INTER & QUERY -.-> JS
  SNAP -.-> RENDER
  NET & STOR & CAP -.-> POLICY
  CDPH -.-> JS & RENDER & POLICY

  subgraph Firefox["Firefox 150+ (owned temp profile)"]
    RA[Remote Agent<br/>WebDriver BiDi]
    DS[DevTools server<br/>accessibility actor]
  end

  CLIENT <-->|ws://127.0.0.1:random/session| RA
  DEVT <-->|unix socket in 0700 dir<br/>or loopback port on Windows| DS
  FF -->|spawn| Firefox

Launch sequence

sequenceDiagram
  participant C as CLI
  participant D as Daemon
  participant FF as firefox.rs
  participant S as FirefoxSession
  participant B as Firefox BiDi
  participant R as Firefox DevTools
  C->>D: open --engine firefox URL
  D->>FF: validate options (reject --cdp, --profile, --args, ...)
  FF->>FF: temp profile + user.js prefs, socket dir 0700
  FF->>B: spawn firefox --headless --remote-debugging-port 0 --start-debugger-server sock
  B-->>FF: stderr "WebDriver BiDi listening on ws://..."
  FF->>S: create session
  S->>B: session.new (proxy, prompts: accept alert/beforeunload, ignore others)
  B-->>S: browserVersion (must be >= 150)
  S->>R: connect, parentAccessibilityActor.enable
  S->>B: session.subscribe (log, network, prompts, downloads, contexts)
  D->>B: browsingContext.navigate URL (wait complete)
  D-->>C: title + url
  Note over D,B: close / idle / crash: end session, kill process group, delete profile

Activity

28
brcampos-iaapproved changes6d ago
vercel[bot]reviewed4d ago
Raillyreviewed4d ago
Railly4d ago
  1. on a firefox session, connect or --provider fails with a --cdp error
  2. after window.close() on the active tab, commands fail with "no such frame"
  3. streaming.mdx still says file:// is allowed
vercel[bot]reviewed3d ago
ctateapproved changes3d ago
ctateapproved changes3d ago
ctateapproved changes3d ago
ctateapproved changes3d ago