fix(type): use insertText for framework-compatible input typing

Open

arielconti10wants to merge intofrom

60 commits83 files changed+8600-1712

Summary

Fixes #10, #11

The type action had two issues when targeting form inputs via selector:

resolveTarget received the full step object. Since resolveTarget checks opts.text first, it would try to find an element matching the text-to-type (e.g. "user@example.com") instead of using the CSS selector. This crashes every type step that has both text and selector set, including the upstream form-filling example.
CDP key events do not update framework-controlled inputs. dispatchKeyEvent fires native keyboard events, but frameworks like React use their own synthetic event system for controlled inputs. Characters were dispatched but never appeared in the field.

Additionally, three recording reliability bugs were fixed:

typeText hangs with charDelay: 0. During recording, every typed character called waitForNextTick() to sync with the capture loop. With zero delay, the typing loop blocked before the capture loop could tick — deadlock.
Stale video frames during navigation. captureScreenshot fired immediately after timeline.tick(). During client-side navigation the compositor hadn't finished rendering, producing outdated frames.
Orphaned Chrome processes on interrupt. kill() sent SIGTERM without waiting for exit, and the non-watch record command had no signal handlers. Ctrl+C during recording left chrome-headless-shell processes running.

Changes

Pass only { selector, within } to resolveTarget instead of the full step object
When a selector is provided, type each character using Input.insertText instead of dispatchKeyEvent. insertText goes through the browser's native text input pipeline, which frameworks respond to
The original typeText path is preserved for non-selector type steps
Add Input.insertText to the CDPClient type
Skip waitForNextTick() when charDelay is 0 to prevent deadlock
Add double-rAF sync before captureScreenshot to ensure fresh compositor frames
Make chrome.kill() async with SIGKILL fallback after 3s timeout
Register process.on('exit') cleanup handler at Chrome launch time
Add SIGINT/SIGTERM handlers to the non-watch record command

Reproduction

The upstream form-filling example crashes on main:

cd examples/form-filling
npx webreel record form-filling --verbose

Recording: form-filling
[step 0] pause 500ms
[step 1] type "user@example.com"
Step 1 (type) failed: Element not found: text="user@example.com"

Examples

All HTML pages and webreel configs used below are available in this gist for local reproduction.

Before fix

Uses key steps with the HUD enabled so you can see it tries to type — but nothing appears in the inputs because dispatchKeyEvent doesn't trigger React's onChange.

https://github.com/user-attachments/assets/6327c001-5e59-4f02-be83-cf44ee030f01

Config

{
  "videos": {
    "inserttext-broken": {
      "url": "./web/index.html",
      "viewport": { "width": 1920, "height": 1080 },
      "zoom": 2,
      "waitFor": "#name",
      "thumbnail": { "time": 6 },
      "steps": [
        { "action": "pause", "ms": 800 },
        { "action": "click", "selector": "#name" },
        { "action": "pause", "ms": 300 },
        { "action": "key", "key": "J", "delay": 100 },
        { "action": "key", "key": "a", "delay": 100 },
        { "action": "key", "key": "n", "delay": 100 },
        { "action": "key", "key": "e", "delay": 100 },
        { "action": "key", "key": " ", "label": "Space", "delay": 100 },
        { "action": "key", "key": "D", "delay": 100 },
        { "action": "key", "key": "o", "delay": 100 },
        { "action": "key", "key": "e", "delay": 200 },
        { "action": "click", "selector": "#email" },
        { "action": "pause", "ms": 300 },
        { "action": "key", "key": "j", "delay": 100 },
        { "action": "key", "key": "a", "delay": 100 },
        { "action": "key", "key": "n", "delay": 100 },
        { "action": "key", "key": "e", "delay": 100 },
        { "action": "key", "key": "@", "delay": 100 },
        { "action": "key", "key": "e", "delay": 100 },
        { "action": "key", "key": "x", "delay": 100 },
        { "action": "key", "key": "a", "delay": 100 },
        { "action": "key", "key": "m", "delay": 100 },
        { "action": "key", "key": "p", "delay": 100 },
        { "action": "key", "key": "l", "delay": 100 },
        { "action": "key", "key": "e", "delay": 100 },
        { "action": "key", "key": ".", "delay": 100 },
        { "action": "key", "key": "c", "delay": 100 },
        { "action": "key", "key": "o", "delay": 100 },
        { "action": "key", "key": "m", "delay": 100 },
        { "action": "pause", "ms": 2000 }
      ]
    }
  }
}

After fix

Uses type steps with selector — insertText fires the input event that React needs, text appears correctly.

https://github.com/user-attachments/assets/6f3e3ba0-00fa-4a26-b5c0-77b17edec79b

Config

{
  "videos": {
    "inserttext-fix": {
      "url": "./web/index.html",
      "viewport": { "width": 1920, "height": 1080 },
      "zoom": 2,
      "waitFor": "#name",
      "defaultDelay": 300,
      "thumbnail": { "time": 6 },
      "steps": [
        { "action": "pause", "ms": 800 },
        { "action": "type", "text": "Jane Doe", "selector": "#name", "charDelay": 60 },
        { "action": "type", "text": "jane@example.com", "selector": "#email", "charDelay": 40 },
        { "action": "pause", "ms": 2000 }
      ]
    }
  }
}

With `charDelay: 0`

Previously, charDelay: 0 would deadlock during recording because typeText blocked on waitForNextTick() faster than the capture loop could tick. Now it completes instantly.

https://github.com/user-attachments/assets/7ae37ee8-0e0e-45e6-b956-49492fc3cdfc

Config

{
  "videos": {
    "chardelay-zero": {
      "url": "./web/index.html",
      "viewport": { "width": 1920, "height": 1080 },
      "zoom": 2,
      "waitFor": "#name",
      "defaultDelay": 300,
      "thumbnail": { "time": 4 },
      "steps": [
        { "action": "pause", "ms": 800 },
        { "action": "type", "text": "Jane Doe", "selector": "#name", "charDelay": 0 },
        { "action": "type", "text": "jane@example.com", "selector": "#email", "charDelay": 0 },
        { "action": "pause", "ms": 2000 }
      ]
    }
  }
}

Stale video frames during navigation

Without the double-rAF sync, captureScreenshot fires immediately after timeline.tick(). During page navigation the compositor hasn't finished rendering, so the capture loop hits consecutive errors and aborts. The test navigates between 4 visually distinct pages (black/Home, blue/Features, purple/Pricing, green/About) 8 times.

	Fixed	Broken (without double-rAF)
Duration	8.25s (495 frames)	1.70s (102 frames)
Pages captured	All 8 transitions	Home only, then capture loop dies

After fix — all 8 page transitions captured cleanly: https://github.com/user-attachments/assets/a1a4efb9-1811-4e04-b636-46cde5f30357

Before fix — capture loop aborts after the first navigation, producing a 1.7s video stuck on the home page: https://github.com/user-attachments/assets/831ceca6-6a49-4928-ad1c-c827f031b5b3

Config

Both configs use the same steps — 8 navigate actions cycling through 4 separate HTML pages with dramatically different background colors.

{
  "videos": {
    "stale-frames-fix": {
      "url": "./web/home.html",
      "viewport": { "width": 1920, "height": 1080 },
      "zoom": 2,
      "waitFor": "nav",
      "defaultDelay": 200,
      "steps": [
        { "action": "pause", "ms": 1000 },
        { "action": "navigate", "url": "./web/features.html", "delay": 800 },
        { "action": "navigate", "url": "./web/pricing.html", "delay": 800 },
        { "action": "navigate", "url": "./web/about.html", "delay": 800 },
        { "action": "navigate", "url": "./web/home.html", "delay": 800 },
        { "action": "navigate", "url": "./web/pricing.html", "delay": 800 },
        { "action": "navigate", "url": "./web/features.html", "delay": 800 },
        { "action": "navigate", "url": "./web/about.html", "delay": 800 },
        { "action": "navigate", "url": "./web/home.html", "delay": 600 },
        { "action": "pause", "ms": 1000 }
      ]
    }
  }
}

Orphaned Chrome processes

On main, sending SIGINT during a recording leaves 4 orphaned chrome-headless-shell processes. This fix registers a process.on('exit') cleanup handler at Chrome launch time and adds SIGINT/SIGTERM handling to the record command.

BEFORE FIX (main @ fec3c34)
========================================
=== Starting recording (will interrupt after 3s) ===
Recording: long-pause
[step 0] pause 60000ms

=== Sending SIGINT to webreel (PID 42085) ===

=== Checking for orphaned chrome-headless-shell processes ===
FAIL: 4 orphaned Chrome process(es) found
  PID 42089 chrome-headless-shell
  PID 42087 chrome-headless-shell
  PID 42094 chrome-headless-shell
  PID 42090 chrome-headless-shell


AFTER FIX (fix/type-inserttext @ 26992dc)
========================================
=== Starting recording (will interrupt after 3s) ===
Recording: long-pause
[step 0] pause 60000ms

=== Sending SIGINT to webreel (PID 42337) ===

Interrupted. Cleaning up...

=== Checking for orphaned chrome-headless-shell processes ===
PASS: No orphaned Chrome processes

Reproduction script

#!/bin/bash
# test-chrome-cleanup.sh <path-to-webreel-cli>
# Starts a long recording, sends SIGINT after 3s, checks for orphaned Chrome.

CLI="${1:?Usage: $0 <path-to-webreel-cli>}"
DIR="$(cd "$(dirname "$0")" && pwd)"

CHROME_PATTERN=".webreel/bin/chrome-headless-shell"

pkill -f "$CHROME_PATTERN" 2>/dev/null || true
sleep 1

echo "=== Starting recording (will interrupt after 3s) ==="
node "$CLI" record -c "$DIR/webreel.config.json" --verbose &
PID=$!

sleep 3
echo ""
echo "=== Sending SIGINT to webreel (PID $PID) ==="
kill -INT "$PID" 2>/dev/null || true

sleep 4

echo ""
echo "=== Checking for orphaned chrome-headless-shell processes ==="
if ps aux | grep "$CHROME_PATTERN" | grep -v grep > /dev/null 2>&1; then
  echo "FAIL: orphaned Chrome processes found"
  ps aux | grep "$CHROME_PATTERN" | grep -v grep | awk '{print "  PID", $2}'
  pkill -f "$CHROME_PATTERN" 2>/dev/null || true
  exit 1
else
  echo "PASS: No orphaned Chrome processes"
  exit 0
fi

With this config:

{
  "videos": {
    "long-pause": {
      "url": "data:text/html,<h1>Test</h1>",
      "viewport": { "width": 1280, "height": 720 },
      "steps": [{ "action": "pause", "ms": 60000 }]
    }
  }
}

Test plan

pnpm test passes (216 tests)
pnpm build clean across all packages
form-filling example records successfully
Tested against a React controlled input form (character-by-character animation visible, form submits correctly)
charDelay: 0 recording completes without hanging
Ctrl+C during recording leaves no orphaned Chrome processes
Navigation recording captures all page transitions (8 navigations between 4 distinct pages)
Tested against a production Next.js app with React controlled inputs

Activity

fix(type): use insertText for framework-compatible input typing

0f0f201Mar 5

vercel[bot]8mo ago

@arielconti10 is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

fix(recording): resolve typeText hang, stale frames, and orphaned Chrome

26992dcMar 5

vercel[bot]reviewed8mo ago

fix(chrome): prevent kill() hang when process already exited

b14d5c8Mar 5

vercel[bot]reviewed8mo ago

ctate8mo ago

Thanks for this excellent contribution! The root cause analysis, before/after videos, and test coverage are all really well done. The insertText approach for framework-compatible typing is the right call.

One thing to fix before merging: the two empty catch {} blocks in chrome.ts (lines 273 and 276) will fail CI's ESLint no-empty rule. Adding a short comment inside each one (e.g. // ignore) should do it.

Everything else looks good — build, type-check, and all 215 tests pass cleanly.

fix: annotate chrome cleanup catches

2c35fe9Mar 7

arielconti108mo ago

Thanks for this excellent contribution! The root cause analysis, before/after videos, and test coverage are all really well done. The insertText approach for framework-compatible typing is the right call.

One thing to fix before merging: the two empty catch {} blocks in chrome.ts (lines 273 and 276) will fail CI's ESLint no-empty rule. Adding a short comment inside each one (e.g. // ignore) should do it.

Everything else looks good — build, type-check, and all 215 tests pass cleanly.

Thanks! It's my pleasure to contribute to this awesome project! I've added the comments!

feat(dsl): add upload action for file input via CDP setFileInputFiles

d43ce1bMar 11

Merge branch 'advisor/014-config-validation-single-source' into fix/type-inserttext

7309b3cJul 19

Merge branch 'advisor/005-hud-overlay-escaping' into fix/type-inserttext

62d4e45Jul 19

Merge branch 'advisor/004-ws-override-node-engines' into fix/type-inserttext

413d377Jul 19

Merge branch 'advisor/003-watch-mode-overlap-guard' into fix/type-inserttext

5a8b21bJul 19

Merge branch 'advisor/002-cdp-error-disconnect-handlers' into fix/type-inserttext