#2203

fix(keyboard): send code and keyCode on keydown and keyup

Openosrjvwants to merge intofrom

keydown and keyup sent only key to Input.dispatchKeyEvent, so pages saw code == "" and keyCode == 0 for held keys. Games, canvas apps and shortcut libraries that read event.code could not be driven with held keys, and press can't hold one. keydown on a letter also typed nothing into a focused field. press Shift was broken too: named_key_info had no modifier entries, so it sent the invalid code "Shift", which Chrome drops.

keydown and keyup now look the key up with named_key_info like press does, and send code, windowsVirtualKeyCode and location, plus text on keyDown for printable keys. named_key_info maps Shift, Control, Alt and Meta to their left-hand keys (ShiftLeft/16, ControlLeft/17, AltLeft/18, MetaLeft/91), accepts the ...Right variants, and accepts code-style names (KeyA-KeyZ, Digit0-Digit9) as Playwright does. A modifier's keyDown sets its own bit, so keydown Shift and press Shift report shiftKey: true. The keydown/keyup path does not send nativeVirtualKeyCode (#2053); press is left to #2112.

Behavior change: a held printable key now types into the focused field, and a held ArrowUp moves the caret, as on a real keyboard and with Playwright's keyboard.down.

Repro

<input id="i" autofocus>
<script>
  window.__k = [];
  for (const t of ["keydown", "keypress", "keyup"])
    addEventListener(t, e => __k.push(`${t} key=${e.key} code=${e.code} keyCode=${e.keyCode} shift=${e.shiftKey}`), true);
</script>
agent-browser open file:///tmp/keys.html
agent-browser keydown w;       agent-browser keyup w
agent-browser keydown Shift;   agent-browser keyup Shift
agent-browser keydown ArrowUp; agent-browser keyup ArrowUp
agent-browser press d
agent-browser press Shift
agent-browser eval "__k.join('\n') + '\nvalue=' + document.getElementById('i').value"

Before (0.38.1, same code on main)

keydown key=w code= keyCode=0 shift=false
keyup key=w code= keyCode=0 shift=false
keydown key=Shift code= keyCode=0 shift=false
keyup key=Shift code= keyCode=0 shift=false
keydown key=ArrowUp code= keyCode=0 shift=false
keyup key=ArrowUp code= keyCode=0 shift=false
keydown key=d code=KeyD keyCode=68 shift=false
keypress key=d code=KeyD keyCode=100 shift=false
keyup key=d code=KeyD keyCode=68 shift=false
keydown key=Shift code= keyCode=0 shift=false
keyup key=Shift code= keyCode=0 shift=false
value=d

After

keydown key=w code=KeyW keyCode=87 shift=false
keypress key=w code=KeyW keyCode=119 shift=false
keyup key=w code=KeyW keyCode=87 shift=false
keydown key=Shift code=ShiftLeft keyCode=16 shift=true
keyup key=Shift code=ShiftLeft keyCode=16 shift=false
keydown key=ArrowUp code=ArrowUp keyCode=38 shift=false
keyup key=ArrowUp code=ArrowUp keyCode=38 shift=false
keydown key=d code=KeyD keyCode=68 shift=false
keypress key=d code=KeyD keyCode=100 shift=false
keyup key=d code=KeyD keyCode=68 shift=false
keydown key=Shift code=ShiftLeft keyCode=16 shift=true
keyup key=Shift code=ShiftLeft keyCode=16 shift=false
value=dw

value=dw because ArrowUp moved the caret to the start before press d.

Tested

  • New e2e test e2e_keydown_keyup_send_code_and_key_code checks the full event log (key, code, keyCode, location, shiftKey) and the input value for keydown/keyup on w, Shift, KeyD and ArrowUp, plus press Shift. It fails on main with code "" and keyCode 0 and passes here (headless Chrome 156, macOS).
  • New unit tests for named_key_info (modifiers, code-style names, near misses), the modifier bit, and the exact Input.dispatchKeyEvent params for keydown/keyup, which pins the missing nativeVirtualKeyCode.
  • cargo test: 1,471 unit and 23 CLI integration tests pass. cargo fmt -- --check and cargo clippy -- -D warnings are clean.
  • cargo test --profile ci e2e -- --ignored --test-threads=1 with ffmpeg and Obscura v0.2.4: 156 passed, 0 failed. The other 4 ignored tests pass too.
  • pnpm --filter docs test, type-check, build and test:routes pass.

Docs: --help, README, the core skill and its commands reference, and the Commands docs page. MCP keydown/keyup delegate to the CLI command; their key description drops the Control+a example, since these commands don't parse chords.

Related

  • #2197 sends modifier key events for chords through named_key_info("Control"), so it needs the modifier entries added here. It also adds a dispatch_key_event helper; whichever PR lands second should merge it with the keydown/keyup helper here.
  • #2103 carries held modifiers from keydown to later mouse commands and touches handle_keydown/handle_keyup.
  • #2112 drops nativeVirtualKeyCode from press (#2053). The keydown/keyup path here already omits it.

Activity

2
vercel[bot]7d ago

@osrjv is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.