Module ui_handlers 

Agent-facing UI automation (UIScreenshot / UIClick / UIQuery, and the browser-pane deep-control tools BrowserNavigate / BrowserBack / BrowserForward / BrowserReload / BrowserEval / BrowserDispatchKey / BrowserFocusElement / BrowserFocusInfo) — proxies to the paired CEF host’s /agentmux/browser/* CDP routes (crates/cef/src/browser_api/). See docs/specs/SPEC_AGENT_BROWSER_PANE_DEEP_CONTROL_2026_09_20.md for the browser-pane tools’ own design (they additionally require the caller’s own pane to be a dedicated browser pane — enforced host-side, not here).

block_id is never a client-supplied field on any request here (2026-08-19, reagent + Codex review, PR #2662 — a bare client-supplied block_id was a real cross-agent content-disclosure vulnerability: /api/v1/ui/* shares the same instance-wide X-AuthKey every App-API route trusts, and any agent can read that key from its own environment, so no amount of “the MCP schema never exposes it” convention actually stopped a bypassing agent from supplying a different pane’s real block_id). Every request instead carries UiAutomationAuth — an HMAC-SHA256 signature over the caller’s own agent_id, using that agent’s own AGENTMUX_JEKT_KEY (the same per-agent key already used for jekt sender authentication, see agentmux_common::jekt_sign). verified_block_id verifies that signature against the claimed agent’s key on file, and — only once verification succeeds — looks up that agent’s actual current block_id server-side via the global ReactiveHandler registry. The block_id a call actually operates on is never taken from the client at all, so there’s nothing left to spoof: an agent without a valid signature for identity X cannot act as X, full stop, regardless of what it claims in the request body.

See agentmux_common::api_types’s “UI automation” section and docs/specs/SPEC_AGENT_UI_AUTOMATION_CLICK_SCREENSHOT_2026_08_18.md.

Constants

POPUP_CHAIN_LIMIT 🔒

How many openers up a popup’s chain the checks follow. A chain this deep is a page opening popups from popups on purpose; driving stops there.

SCREENSHOT_RETENTION 🔒

Delete *.png files in dir older than SCREENSHOT_RETENTION. Called on every UIScreenshot write since this directory only ever grows otherwise (reagent P2, PR #2662) — no background scheduler needed for a directory nothing else writes to. Best-effort: any I/O error for an individual entry (permissions, a concurrent delete, a non-UTF8 name) is skipped rather than failing the screenshot request that triggered it.

UI_AUTOMATION_SIG_MAX_AGE_SECS 🔒

Signatures older than this are rejected — bounds replay of a leaked/ logged signature to a short window. Mirrors server/reactive.rs’s JEKT_SIG_MAX_AGE_SECS (same threat model: a host-tier HMAC signature verified locally, not a one-time nonce scheme).

Functions

banner_for 🔒

Where a request to the person about target shows: in its own banner, or, for a popup window (which has no AgentMux header), in its opener’s, with the window’s address so the banner can say which window it means.

chain_root 🔒

The pane a chain of popups started in: pane itself unless it is a popup pane, else its opener’s root, as far up as the panes still exist. Popups are counted over its whole tree, so a popup opening popups shares its root’s cap rather than getting its own.

err_response 🔒

from_host 🔒

Is this request from the CEF host? X-Host-Token must be the IPC token the host registered with (host_ipc.Register), which agents never see; the instance auth key alone, which every agent has, isn’t enough.

get_host_ipc 🔒

handle_host_browser_attention 🔒

POST /api/v1/host/browser_attention — the user’s answer to a hand-off or approval banner, relayed by the CEF host. Only the host can call it: X-Host-Token must be the IPC token the host registered with (host_ipc.Register), which agents never see; the instance auth key alone, which every agent has, isn’t enough.

handle_host_browser_popup 🔒

POST /api/v1/host/browser_popup — a browser pane’s page opened a popup (docs/specs/SPEC_BROWSER_PANE_POPUPS_ADOPTED_2026_10_08.md). Only the host calls it, from on_before_popup. Answers {admitted: true, pane} when srv opened the popup as a pane beside its opener, or {admitted: false, reason}, and the host opens it in the system browser as it always did.

handle_host_browser_popup_window 🔒

POST /api/v1/host/browser_popup_window — the host created, navigated or closed a popup window (a native window a browser pane’s page opened; SPEC_BROWSER_PANE_NATIVE_POPUPS_AGENT_DRIVEN_2026_10_08.md §4). Body {event: "opened"|"navigated"|"closed", popup, opener, url}. Only the host can call it. The window’s owner is the opener’s, from srv’s own record; the opener’s popup-windows strip is kept current.

handle_ui_browser_act 🔒

POST /api/v1/ui/browser/act — backs BrowserClick, BrowserFill, BrowserSelect and BrowserCheck (spec §4.2).

handle_ui_browser_back 🔒

POST /api/v1/ui/browser/back — backs BrowserBack.

handle_ui_browser_dispatch_key 🔒

POST /api/v1/ui/browser/dispatch_key — backs BrowserDispatchKey.

handle_ui_browser_eval 🔒

POST /api/v1/ui/browser/eval — backs BrowserEval. Returns the host’s {result, type, exception} shape verbatim under data, same passthrough pattern handle_ui_query uses for matches.

handle_ui_browser_focus_element 🔒

POST /api/v1/ui/browser/focus_element — backs BrowserFocusElement.

handle_ui_browser_focus_info 🔒

POST /api/v1/ui/browser/focus_info — backs BrowserFocusInfo. Returns the host’s {focused} shape verbatim under data.

handle_ui_browser_forward 🔒

POST /api/v1/ui/browser/forward — backs BrowserForward.

handle_ui_browser_handoff 🔒

POST /api/v1/ui/browser/handoff — backs BrowserHandoff (spec §5.2): a banner in the pane asks the user to do something the agent mustn’t (sign in, a CAPTCHA), and the call waits for their Done or Cancel. The agent’s tools on that pane are refused meanwhile (target_block_id).

handle_ui_browser_navigate 🔒

POST /api/v1/ui/browser/navigate — backs BrowserNavigate. Own pane only, and only when it’s a dedicated browser pane (checked host-side).

handle_ui_browser_open 🔒

POST /api/v1/ui/browser/open — backs OpenBrowser. Opens a browser pane next to the caller’s own pane and records the caller as its owner, so the Browser*/UI* tools accept it as pane (docs/specs/SPEC_AGENT_DRIVEN_BROWSER_PANES_2026_10_07.md §3).

handle_ui_browser_reload 🔒

POST /api/v1/ui/browser/reload — backs BrowserReload.

handle_ui_browser_set_files 🔒

POST /api/v1/ui/browser/set_files — backs BrowserSetFiles. Only files inside the calling agent’s own workspace (its block’s launch directory, host agents only) are allowed (spec §5.3).

handle_ui_browser_snapshot 🔒

POST /api/v1/ui/browser/snapshot — backs BrowserSnapshot (docs/specs/SPEC_AGENT_DRIVEN_BROWSER_PANES_2026_10_07.md §4.1).

handle_ui_browser_wait_for 🔒

POST /api/v1/ui/browser/wait_for — backs BrowserWaitFor.

handle_ui_click 🔒

POST /api/v1/ui/click — backs the UIClick MCP tool. Synthesizes a real mouse click at the first selector match within the caller’s own pane subtree.

handle_ui_query 🔒

POST /api/v1/ui/query — backs the UIQuery MCP tool. Returns matched elements (tag/text/attrs/rect/focused) within the caller’s own pane subtree — the same shape browser_api::types::QueryData returns.

handle_ui_screenshot 🔒

POST /api/v1/ui/screenshot — backs the UIScreenshot MCP tool. Captures a PNG clipped to the caller’s own pane, writes it to <mux_data_dir>/tmp/ui-screenshots/<uuid>.png, and returns both the path (openable via OpenMedia) and the base64 bytes inline.

not_waiting_on_user 🔒

Refuse a tool call on a pane that is waiting for the user: a hand-off (they’re working in it) or an approval (they’re reading the form the approved click will send, which mustn’t change under them).

opener_allows 🔒

A popup pane (browser:popup_of) is driven as part of its opener: while the opener exists, the agent must still own it (the person’s Take over on the opener ends its hold on the popups it opened too), and the opener must not be waiting on the person (a hand-off there pauses its popups). A popup whose opener was closed stands on its own.

owning_agent 🔒

The agent that owns block_id, if one does and the person hasn’t taken the pane over: the same test every pane argument passes.

popup_listing 🔒

opener’s open popups for its snapshot: each one’s pane id, address, and whether agent_id may drive it (not if the person took it over).

popup_of 🔒

The pane that opened block as a popup pane, if it is one.

popup_window_target 🔒

A popup window (a native window a pane’s page opened, id popup-…) may be driven by the agent recorded when it opened, while that agent still owns the pane that opened it and that pane isn’t waiting on the person (SPEC_BROWSER_PANE_NATIVE_POPUPS_AGENT_DRIVEN_2026_10_08.md §5).

proxy_ack 🔒

Proxies route with body and checks the host’s {ok, ...} envelope. Shared by the write-only browser-pane tools (navigate/back/forward/ reload/focus_element/dispatch_key) — they differ only in which route/body they send, not in how they interpret an ack-only reply.

proxy_data 🔒

proxy_to_host 🔒

POST body to /agentmux/browser/{route} on the paired host and return its parsed JSON response (the host’s own {ok, data} / {ok, error} envelope — see browser_api::types::ApiResponse).

proxy_to_host_timeout 🔒

proxy_to_host with a request timeout of its own, for host routes that legitimately take longer than the shared client’s default (a snapshot of a large page, wait_for).

proxy_value 🔒

Proxy a browser-pane request to the host and pass its data through: the shared tail of the snapshot and act handlers.

prune_old_screenshots 🔒

refuse_on_driven_pane 🔒

On a browser pane an agent drives (opened with OpenBrowser), refuse the general-purpose routes that could submit, send or delete without the approval banner (spec §5.4): arbitrary script, clicks by CSS selector, and the keys that press buttons. The reference tools do the same work with the approval check in front of committing clicks.

reset_popup_windows 🔒

Forget every popup window and clear every opener’s strip. Called when the host registers: after either side restarts, the windows srv knew of are gone or no longer known, and the strip, which is saved with the pane, must not keep listing them.

target_block_id 🔒

The block a UI-automation request acts on: the caller’s own pane, or, when pane is given, a browser pane the caller opened with OpenBrowser (docs/specs/SPEC_AGENT_DRIVEN_BROWSER_PANES_2026_10_07.md §3). The caller’s identity is verified first either way; pane is then checked against browser_owner’s map, never trusted on its own.

update_popup_windows_strip 🔒

Rewrite opener’s popup-windows strip (browser:popup_windows). The opener may be closing; then there’s nothing to update.

verified_block_id 🔒

Verify auth proves the caller genuinely is auth.agent_id (via that agent’s own jekt key — an attacker without agent_id’s key cannot produce a valid signature no matter what it claims), then derive that agent’s ACTUAL current block_id server-side. Never trusts a block_id from the client — there isn’t one to trust. See this module’s own doc comment for the full rationale. pub(crate): also reused by app_api::pane::handle_close_pane (and any future own-pane-resolving handler) — the identity-verification mechanism isn’t UI-automation-specific, just first built for it. See docs/specs/SPEC_AGENT_PANE_LIFECYCLE_CONTROL_2026_09_10.md §5.0.