Agent & CLI Reference

SafeCoBrowser is a real, logged-in browser the human controls. Your agent is a guest that can see and act only inside the permission they grant — per tab, right now — and they can cut access instantly. You are the “brain”; SafeCoBrowser is the gated “body.”

• Every tab is Off by default — until a mode is granted, every call returns permission_denied.
• You cannot change the permission mode — only the user’s UI can.
• You act on the active tab; to work on another, bring it to the front with switch_tab first (and you can never open or close tabs). One exception in the next release: selector click/fill with Real input on, in the Electron/Linux build — see Real input under Agent tips.
• You never get cookies, passwords, or the profile — only brokered tools.
• Every call (allowed or denied) is written to a tamper-evident audit log.

On launch SafeCoBrowser starts a localhost server (bound to 127.0.0.1 only) and writes its endpoint + bearer token to ~/.safecobrowser/endpoint.json. The token is required even on localhost; the server also validates Host/Origin.

~/.safecobrowser/endpoint.json
{ "url": "http://127.0.0.1:8676", "token": "<bearer>" }

The MCP endpoint is POST {url}/mcp (Streamable HTTP). Register it with Claude Code:

TOKEN=$(node -pe "JSON.parse(require('fs').readFileSync(require('os').homedir()+'/.safecobrowser/endpoint.json')).token")
claude mcp add --transport http safecobrowser http://127.0.0.1:8676/mcp \
  --header "Authorization: Bearer $TOKEN"

The server name is safecobrowser; it exposes the tools below. Calls are gated by the tab’s mode, and effectful / run_js calls require approval.

Driving it from another computer. The server is loopback-only by default. To connect a remote agent, either tunnel the port (ssh -N -L 8676:127.0.0.1:8676 you@host, or Tailscale — recommended, encrypted) and point at 127.0.0.1 as usual, or enable Settings → Agent connection → “Allow LAN connections” (default off). That rebinds the server to your LAN and shows a LAN MCP URL to use in place of 127.0.0.1. LAN mode is plaintext HTTP — use it only on a trusted network; for the internet keep it off and use the tunnel. The bearer token, the Host/Origin allowlist, and the per-tab AI block all still apply.

The bundled safecobrowser CLI is a thin client over the same broker — handy for scripting and testing.

safecobrowser status                 # session status (+ the active tab id)
safecobrowser tools                  # list tools with their mode / risk / approval
safecobrowser invoke <tool> [json]   # invoke a tool through the broker
safecobrowser audit [n]              # last n audit records (default 20)
safecobrowser audit verify           # verify the audit log's hash chain

status/tools/invoke need SafeCoBrowser running; audit reads the local log and works offline.invoke exits 0 on success, 2 on a denial. Note: the CLI cannot raise the mode either.

An ordered ladder, set per tab by the user. A tool runs only if the tab’s current mode is at least the tool’s minimum.

Read the active tab’s current mode with get_mode (or safecobrowser status) instead of guessing from denials — it’s read-only, you still can’t change it. A few tools sit off this ladder and work at any mode (including Off): get_mode, list_tabs, switch_tab, and submit_feedback — they aren’t page operations.

Inputs are validated by the broker; a bad shape returns invalid_input.

• Coordinate arguments — x, y (and dy, dx for scroll) are CSS-viewport pixels, given as numbers. Plain numeric strings such as "221" are also accepted; anything else ("abc", "1e3") is rejected with invalid_input. Numeric strings are accepted as of 1.2 (Mac App Store). The Electron/Linux download is an older build and still rejects them with expected x to be a finite number — fixed in its next release — which is what an MCP client that does not send typed arguments will hit on every call there. Wherever you can, send real JSON numbers.
• fill never logs the value (it may be a password) — only the selector (#email (value hidden)). Filling does not submit; request the submit as a separate click.
• run_js is full page control — its own grant, the script is shown in the approval card, and every script is logged. Prefer the narrower tools where they suffice.
• screenshot base64 is large — save it to a file, don’t echo it.
• list_recipes / get_recipe read the user’s saved recipes (recorded how-to tutorials) for the current site — read one, then reproduce it with click/fill. Scoped to the active tab’s domain; sensitive field values are withheld.
• Prefer locate over screenshots for finding things. On a real DOM, locate returns an element’s coordinates straight from the page layout by text/selector (fast, no vision pass), with inViewport + obscured flags. Pattern: locate → pick a match with inViewport:true, obscured:false → click_at. If the match is inViewport:false, call scroll_to first (it scrolls the element in and returns its settled coordinates). It’s read-only — it never scrolls or changes the page.
• Coordinate ("computer use") tools — move_to, click_at, scroll, press_key, type_text — deliver real trusted input at a viewport point, for canvas/WebGL apps and pages with no usable DOM. Coordinates are CSS viewport pixels; the approval card shows a screenshot with a crosshair on your target. Prefer the selector tools (click/fill) or locate+click_at when the DOM is usable — a coordinate is a poor approval prompt. Not a stealth feature: real input only, no humanized motion/timing.
• read_screen_text is locate for pixels. When the page has no usable DOM — a canvas/WebGL app, or a page that’s a single image — locate finds nothing. read_screen_text OCRs the visible page (offline, on-device) into words with their CSS-viewport coordinates, so you get a target for click_at/type_text without a screenshot + vision pass. It’s not AI vision (you already see the screenshot) — it’s fast, precise text→coordinate extraction. Read-only, no approval, and the recognized text is never written to the audit. On a real DOM, prefer locate.
• list_tabs / switch_tab let you see the open tabs and bring one to the front (your next calls then target it). Switching is not escalation — it never changes a tab’s mode, so a tab that’s Off still exposes nothing; call get_mode after switching. Both are governed by the user’s Allow agent tab control setting (default on); if it’s off, list_tabs shows only the active tab and switch_tab is denied.

Every effectful tool requires approval — click, fill, scroll_to, the coordinate tools (move_to/click_at/scroll/press_key/type_text), and run_js. You invoke the tool, SafeCoBrowser shows the user an approval card with the concrete effect (the selector, the query for scroll_to, a screenshot + crosshair for the coordinate tools, or the full run_js script), and your call blocks until they answer.

• Approve → the tool runs, you get { ok: true, … }.
• Reject → approval_rejected. No answer in ~120s → approval_required.
• The user may enable per-tab auto-approve (one toggle for click/fill, a separate one for run_js) — approved calls then run without a card, but are still logged. Don’t rely on it being on.
• Real input (you don’t control this): each tab has a user-set Real input toggle. When it’s on, your click/fill are delivered as real, trusted input (the cursor actually moves and clicks; text is typed key by key) instead of synthesized events — which some sites require. You can’t read or set it and it never changes your mode; it only changes how the action is delivered. Your result carries realInput: true|false so you know which path ran, and the audit log tags those actions [real input]. If click reports clicked:false with a note like target obscured, an overlay was covering the point — it will not click the wrong thing; clear it and retry. It is not a stealth/anti-detection feature (no humanized motion or timing) — if a site still blocks real, trusted input, ask the user rather than trying to evade.
  In the next release: with Real input on for a tab, click and fill never silently fall back to a synthetic JS event. If real input can’t be delivered they refuse, and the note says why and that real input is on — for example “… not clicked (real input is on, so no JS fallback)”. Reasons include: the field is obscured, focus wasn’t confirmed, the typing didn’t land, and (macOS) SafeCoBrowser isn’t the frontmost app or the tab isn’t the active tab. type_text and press_key already refuse in these cases. Only the user can turn Real input off for a tab (some editors reject keystrokes); then click/fill use the normal path and the result shows realInput:false.

  • macOS: real input needs SafeCoBrowser to be the active app and the tab to be the active tab — a WebKit limitation, not a window-order one.
  • Electron/Linux: selector click and fill with real input also work on a background tab. The tab’s own grant is the only gate, and the approval card names the tab.
  • Both builds: the coordinate tools (move_to, click_at, scroll, press_key, type_text) stay active-tab-only, and so do the orange click outline and replay frames.

Assume the user is watching. In the Mac App Store app (1.1+), every click and fill briefly outlines its target in orange, drawn by the app over the page — it does not appear in your screenshots or page reads, and you cannot suppress it. In 1.2 and later the user can also replay your session afterwards, frame by frame, from Activity → Replays.

Each effectful step you take (click, fill, click_at, move_to, scroll, scroll_to, press_key, type_text, navigate, run_js) is recorded with up to two frames: one at the moment of the action with the target outlined, and one with the result once the page settles. Replays are watch-only — nothing is re-run. Only allowed actions are recorded: never reads, never denied calls, and nothing from the period before a tab was granted. The privacy filter applies to the frames. Recordings stay on the user’s Mac, are kept for 7 days or the newest 50 sessions, and can be turned off or deleted in Settings. The Electron/Linux download is an older build and does not include Replays or the click outline yet.

There is no tool to read replays. The tool list above is unchanged — do not look for one. Recordings are for the human only.

Two Settings affect how pages load for you. Neither is something you can read or change from an agent tool, and both are described here so you can tell what you are seeing. Both are new in the next release.

Every invocation returns an InvokeResult:

{ ok: true,  output: <tool output> }            // success
{ ok: false, reason: <DenyReason>, message? }   // denial / failure

All refusals fail closed. The reasons:

The user can turn on a privacy filter — their own list of sensitive strings (name, address, account numbers) that SafeCoBrowser replaces with labels like [Name] or[Account]. There is no tool for it and the agent can’t see or change it.

• What you read may be redacted. read_page, inspect_element, screenshot, and even run_js read the rendered DOM, which is where redaction happens — so you’ll see [Name] where the real value was. That’s intentional.
• Don’t try to defeat it. Treat a label as a value you’re not meant to have; if you genuinely need it, ask the user.
• Best-effort, not total. It redacts visible text nodes only — not form-field values, attributes, or network metadata — so a real value may still surface; treat anything sensitive accordingly.

Every brokered decision is appended to ~/.safecobrowser/audit-log.jsonl, hash-chained for tamper-evidence, surviving restart, and viewable live in the in-app Activity panel.

safecobrowser audit 20        # recent records
safecobrowser audit verify    # re-walk the hash chain (detects edits/deletes/reorder)

click/inspect_element log their selector; run_js logs the full script; fill logs the selector but never the value; switch_tab logs the target tab id.

• Look before you leap: use read_page / inspect_element to find real selectors before click/fill.
• Diagnose with Inspect: combine read_console + read_network (look for non-2xx) before proposing a fix; use read_network_body to read the actual XHR/fetch payload (e.g. the JSON behind a feed) — it only captures responses received after AI was granted.
• Fill then submit, deliberately: fill never submits — request the submit as its own approved click.
• Re-read after navigation: a click that navigates makes your prior DOM knowledge stale.
• Respect denials: approval_rejected means “no” — pick a different approach. permission_denied means “raise the mode,” which only the user can do.
• Work across tabs deliberately: list_tabs to find the tab → switch_tab to it → get_mode (a freshly-switched tab is often Off — ask the user to raise it) → then read/act. Don’t hammer a tab you just switched to.
• Minimize run_js: prefer the narrower tools; keep any script tight and readable (the user reads it in the card).
• Follow a recipe’s intent, not just its rows: if get_recipe’s description states a pattern (“click each row’s download button, incrementing the index until none match”), read the live page and apply it to all items — there may be more than were recorded.

# typical loop (CLI form; MCP tool calls are 1:1)
safecobrowser status
safecobrowser invoke read_page
safecobrowser invoke inspect_element '{"selector":"form button[type=\"submit\"]"}'
safecobrowser invoke fill  '{"selector":"#email","value":"[email protected]"}'
safecobrowser invoke click '{"selector":"button[type=\"submit\"]"}'