macOS Computer Use: Unreal Editor coordinate clicks activate the control under the physical cursor

What version of the Codex App are you using (From “About Codex” dialog)?

26.1023.51421 (build 22530), read from installed ChatGPT.app bundle metadata; Codex surface, bundle ID com.openai.codex.

Bundled Computer Use helper: 26.1022.1001641 (build 1001641).

What subscription do you have?

Plus

What platform is your computer?

Darwin 27.0.0 arm64 arm

macOS 27.0.1, build 26A434. Apple Silicon / ARM64.

Target: Unreal Editor 5.8.2, changelist 56702186, Mac Metal.

What issue are you seeing?

A coordinate click requested on Unreal Editor's Outliner search field activates Tools, where the human left the physical cursor. The Computer Use pointer is drawn over the requested search field, but the field does not gain focus. The call completes without a window-location error.

This was reproduced on October 29 after updating Codex, using the documented mcp__cua_repl / cua app API. The UI operator was GPT-6.1 Terra with High reasoning; only one agent operated the UI. The human's role was to park the physical cursor without clicking.

This occurred while working on a UE5 project, using the standard Unreal Editor menu and Outliner.

Control applications work in the same session:

The behavior is not a complete loss of mouse-input capability or screen capture. It blocks autonomous Unreal authoring and drag/drop verification.

What steps can reproduce the bug?

1. Open Unreal Editor on macOS and make the standard menu and Outliner visible. Put the Unreal window in front.
2. Bind it using the exact UnrealEditor.app path to avoid an ambiguous app identifier.
3. Have the human park the physical cursor over Tools without clicking, then stop touching the mouse.
4. Capture the initial state: Tools has gray hover highlighting.
5. Request a Computer Use coordinate click on the Outliner search field, a different location.
6. Capture the result: the virtual pointer is over Outliner, but Tools turns blue/active, and the search field does not gain focus.
7. Send Escape: Tools returns to gray.

Recorded action, for the original 3024 × 1736 screenshot/window layout:

// The human has parked the physical cursor over Tools without clicking.
// ue is the app bound through the documented cua.getApp API.
await ue.getAXState();
await ue.getScreenshot();
await ue.click([2100, 304]); // Outliner search in this specific capture
await ue.getAXState();
await ue.getScreenshot();
await ue.pressKey("Escape");
await ue.getAXState();
await ue.getScreenshot();

Derive fresh target coordinates for a different window layout or image size. The recorded capture shows the Tools header's active state; it does not establish that a complete menu popup is visible in that main-window screenshot.

The cross-position sequence was recorded on October 27 and repeated on October 29. This is qualitative evidence from those sequences, not a failure-rate benchmark. No native app-side event logger was added.

What is the expected behavior?

The requested Outliner field should gain focus regardless of where the human cursor was parked. Tools should remain inactive.

Mouse movement/down/up and target-window resolution should agree on the requested position. If reliable coordinate delivery is unsupported for this window, an explicit error is preferable to a successful tool response that silently activates a different control.

Please test a fix with at least two different parked physical-cursor positions, then verify selection and drag/drop as well as simple clicks.

Additional information

Technical diagnosis: observed versus inferred

Observed: the requested coordinate and virtual pointer position disagree with the activated control, which coincides with the physical cursor location prepared by the human.

Verified by inspecting the installed UE 5.8.2 source:

This is consistent with a mismatch between the requested/virtual position and the global/cached cursor position consumed by Unreal/Slate. Applications using a different event-position path may still accept coordinate clicks.

The installed JavaScript adapter passes x/y through to its service call. We have not inspected the native service source and do not claim a specific native posting API, a definitive allocation of responsibility between components, or an identified introducing commit. Please route to native macOS Computer Use/input-delivery maintainers.

Earlier window-lookup errors are a separate symptom

Some historical attempts returned:

Computer Use server error -10005: windowNotFoundAtPosition((1110.0, 183.5))

That error did not occur during the October 29 cross-position reproduction. We have not established that silent wrong-control activation and window lookup failure have one root cause.

Attempts and limitations

• Earlier CUA reset/rebinding, full-path app binding, window resizing and held clicks of about 300 ms did not restore reliable Slate clicks.
• A Mac restart and later app updates did not remove the recorded symptom. These are historical attempts, not a full factorial test on build 22530.
• Native accessibility Raise/minimize/zoom and some keyboard navigation worked. Manual focus sometimes allowed text injection, but cannot replace autonomous drag/drop.
• No real/global-cursor or exclusive-foreground selector was found in the documented macOS API/configuration. Computer Use correctly refused to control Codex itself for safety; the human separately checked Settings → Computer Use and confirmed no such option. No restriction was bypassed, mode activated, permissions reset, or Engine/project code patched.
• Stage Manager, multiple Spaces and controller fullscreen were not systematically varied in this reproduction. Those conditions should not be inferred solely from screenshots.
• No controlled older-version A/B was performed. Historical log analysis identifies build 21421 before October 24, transition to 21828/cua_repl around October 24, and first project failure evidence on October 25. Build 22391 was installed later on October 27; current build 22530 still reproduces. This chronology does not prove which release introduced the mismatch. The user reports that Computer Use previously worked.

Related reports

Closest precedent: #45859, SDL coordinate input at the physical cursor. This report adds a separate Unreal/Slate reproduction, successful Finder/Calculator controls, and target-side source analysis. It may share a backend cause; please consolidate if appropriate.

Other related input/window reports: #57020, #56940, #57301, #56216 and #55353. They are not asserted to be duplicates. In particular, the author's correction on #57020 concerns the target being in the current Space, not Stage Manager alone. All six records were open when checked on October 29; no official fix is claimed.