# SafeCoBrowser — AI Agent Guide

> How an external AI agent (Claude Code, Codex, or any MCP client) drives **SafeCoBrowser**:
> the tools, the permission model, how a call resolves, and the patterns that work.
>
> **One-sentence model:** SafeCoBrowser is a real, logged-in browser that the **human controls**.
> You (the agent) are a guest who can see and act **only inside the permission the human
> grants, per tab, right now** — and they can cut your access instantly. You are the "brain";
> SafeCoBrowser is the gated "body."

---

## 1. The golden rules (read first)

1. **You start with nothing.** Every tab is **Blocked** by default. Until the human raises the
   mode in the SafeCoBrowser UI, *every* tool call returns `permission_denied`.
2. **You cannot escalate yourself.** There is **no** tool/endpoint to change the permission
   mode. Only the human's UI can. Don't look for one; it doesn't exist by design.
3. **You act on the ACTIVE tab only.** You cannot choose or target a background tab. Whatever
   tab the human is currently viewing is the tab your calls hit.
4. **Access can vanish mid-call.** "Stop AI" or a mode change bumps a session epoch and makes
   your in-flight and future calls **fail closed** (`revoked` / `permission_denied`). Handle it
   gracefully; never assume a grant persists.
5. **You never get the profile.** No cookies, no password DB, no profile directory, no other
   sites' sessions. You get **only** the brokered tools below. (`run_js` can *read* page-context
   `document.cookie` if granted Developer mode — see §5.8 — but you never touch the cookie store.)
6. **Everything you do is logged.** Every call (allowed or denied) is written to a tamper-evident
   audit log the human can inspect. Act like it's on the record — it is.
7. **No retroactive leak.** When the human grants Read, you see the page **as it is at call
   time** — never what it showed while you were blind (e.g. their login). Don't ask for it.
8. **Some text may be redacted.** If the human's **privacy filter** is on, sensitive values appear
   as labels like `[Name]` / `[Account]` in everything you read. That's deliberate — work with the
   placeholder and never try to recover the hidden value (see §9).

---

## 2. Connecting

SafeCoBrowser runs a localhost control server (bound to `127.0.0.1` only) the moment the app
launches. There are two equivalent front-ends over the **same broker**: MCP and a CLI.

### 2.1 Discover the endpoint
On launch SafeCoBrowser writes:

```
~/.safecobrowser/endpoint.json   →   { "url": "http://127.0.0.1:8676", "token": "<bearer>" }
```

The bearer token is **required even on localhost**. The server also validates `Host`/`Origin`
against DNS-rebinding. If the file is missing, SafeCoBrowser isn't running.

### 2.2 MCP (for Claude Code / Codex)
- **Endpoint:** `POST {url}/mcp` — MCP **Streamable HTTP**, stateless (one server/transport per
  request), `enableJsonResponse: true`.
- **Auth header:** `Authorization: Bearer <token>` (from `endpoint.json`).
- **Server name:** `safecobrowser`. It exposes exactly the tools in §5; each MCP tool call is
  forwarded straight to the broker.
- The MCP input schema is intentionally permissive (open object) — the **broker** does the real
  per-tool validation, so malformed input comes back as `invalid_input`, not a schema crash.

Example Claude Code registration (conceptual — read the live token from the file):
```jsonc
{
  "mcpServers": {
    "safecobrowser": {
      "transport": "http",
      "url": "http://127.0.0.1:8676/mcp",
      "headers": { "Authorization": "Bearer <token-from-~/.safecobrowser/endpoint.json>" }
    }
  }
}
```

### 2.3 CLI (same broker, handy for scripting/testing)
```
safecobrowser status                 # session status (and the active tab id)
safecobrowser tools                  # list available tools + their mode/risk/approval
safecobrowser invoke <tool> [json]   # invoke a tool through the broker
safecobrowser audit [n]              # last n audit records (default 20)
safecobrowser audit verify           # verify the audit log's hash chain
```
(Run as `node dist/cli/safecobrowser.js …` until a `bin` is installed.)
`status`/`tools`/`invoke` need SafeCoBrowser running; `audit` reads the local log and works offline.

A tool call returns a JSON **InvokeResult** (see §6). The CLI exits `0` on `ok`, `2` on denial.

---

## 3. The permission ladder

Modes are an ordered ladder. A tool declares the **minimum** mode it needs; the broker allows a
call only if the tab's **current** mode is at least that high.

| Mode | Value | Unlocks (cumulative) |
|------|-------|----------------------|
| **Blocked** | `blocked` | nothing — AI is off (default) |
| **Read** | `read` | `read_page`, `screenshot`, `list_recipes`, `get_recipe` |
| **Inspect** | `inspect` | + `inspect_element`, `read_console`, `read_network` |
| **Act** (UI: *Assist*) | `act` | + `click`, `fill` (always behind approval) |
| **Develop** (UI: *Developer*) | `develop` | + `run_js` (full page control) |

- Higher modes include everything below (Act can also Read/Inspect).
- The human sets this **per tab** in the toolbar. Note: the control API does **not** expose the
  current mode directly — `safecobrowser status` returns only `{ ok, tab }` (the active tab id). You
  infer the mode from results: a `permission_denied` means the tab is below that tool's `minMode`.
- **Do not nag.** If you're denied, state what mode you'd need and let the human decide.

---

## 4. What you can and can't do at each mode

- **Blocked:** nothing. Every call → `permission_denied`.
- **Read:** summarize/extract the page, screenshot it, and **read the user's saved recipes** for
  this site (`list_recipes` / `get_recipe`) to learn how a task is done here. No DOM queries, no
  console/network, no actions.
- **Inspect:** everything in Read **plus** element inspection and console/network metadata — ideal
  for *diagnosing* a page (find selectors, read errors, see failed requests) without changing it.
- **Act / Assist:** everything above **plus** `click` and `fill`, each gated by an **approval
  card** (or the human's per-tab auto-approve). Use this to drive forms/buttons.
- **Develop:** everything **plus** `run_js` — arbitrary JS in the page. Most powerful and most
  dangerous; its own explicit grant, always approval-gated, always logged.

---

## 5. The tools (full reference)

> Inputs are validated by the broker. Bad shapes return `invalid_input` (fail closed).
> Outputs below are the `output` field of a successful InvokeResult.

### 5.1 `read_page` — Read · Low risk · no approval
Read the current page.
- **Input:** none (`{}`)
- **Output:** `{ url, title, text, links: [{ href, text }] }`
- `text` is visible text (whitespace-collapsed); `links` are anchor hrefs + their text.

### 5.2 `screenshot` — Read · Low risk · no approval
Capture the current page as a PNG.
- **Input:** none (`{}`)
- **Output:** `{ mimeType: "image/png", base64 }`
- **Note:** the base64 is large — save it to a file rather than echoing it into the conversation.

### 5.3 `inspect_element` — Inspect · Low risk · no approval
Inspect one DOM element by CSS selector. (Selector is audited.)
- **Input:** `{ selector: string }`
- **Output:** `{ matched, tagName?, attributes?, text?, outerHTML?, rect? }`
- If nothing matches, `matched: false`. Use this to discover real selectors before acting.

### 5.4 `read_console` — Inspect · Low risk · no approval
Recent console messages.
- **Input:** `{ limit?: number }` (default `100`, max `1000`)
- **Output:** `[{ level, text, ts }]`

### 5.5 `read_network` — Inspect · Low risk · no approval
Recent network request **metadata** (not bodies).
- **Input:** `{ limit?: number }` (default `100`, max `1000`)
- **Output:** `[{ method, url, status?, ts }]`
- Great for diagnosing failed submits/API calls (look for non-2xx `status`).

### 5.6 `click` — Act · Medium risk · **approval required**
Click a visible element by CSS selector.
- **Input:** `{ selector: string }`
- **Output:** `{ clicked, matched }` — `matched` is the element's text/label.
- Blocks until the human approves (or auto-approve is on for actions). Selector is audited.

### 5.7 `fill` — Act · Medium risk · **approval required**
Type a value into an input by CSS selector.
- **Input:** `{ selector: string, value: string }`
- **Output:** `{ filled, matched }`
- **The value is NEVER logged** (could be a password); the **selector IS** logged
  (`#email (value hidden)`), so the human can see *what field* you filled without exposing the
  value. Filling does **not** submit — submitting is a separate `click` you must request explicitly.

### 5.8 `run_js` — Develop · **High risk** · **approval required**
Run arbitrary JavaScript in the page and return its result.
- **Input:** `{ script: string }` (max 100,000 chars)
- **Output:** the script's return value (any JSON-serializable value)
- This is **full page control**: it can read `document.cookie` in page context, mutate the DOM,
  and exfiltrate. It is its own grant (never bundled into read-only access), the **script is shown
  to the human in the approval card**, and **every script is logged in full**. Prefer the
  narrower tools (Read/Inspect/Act) whenever they suffice; reach for `run_js` only when you must.

### 5.9 `list_recipes` — Read · Low risk · no approval
List the user's saved **recipes** (recorded, named how-to tutorials) for the current page's site.
- **Input:** none (`{}`)
- **Output:** `{ domain, recipes: [{ name, description?, steps }] }`
- Scoped to the **active tab's registrable domain** (e.g. on `mobile.x.com` you get `x.com`'s
  recipes). Empty list = no recipes for this site. Read-only; this never runs anything.

### 5.10 `get_recipe` — Read · Low risk · no approval
Read one recipe's annotated, step-by-step how-to so you can reproduce it with `click`/`fill`.
- **Input:** `{ name: string }` (a name from `list_recipes`)
- **Output:** `{ name, domain, description?, steps: [{ n, name?, description?, action, type, selector?, label?, value?, sensitive?, url? }] }`
- `action` is a human summary (e.g. `type "askmingli" into "Search"`, `press Enter to submit`);
  `selector`/`label` let you re-perform the step with your own tools. **Sensitive field values are
  withheld** (`sensitive: true`, no `value`) — you must ask the human to fill those.
- Pattern: on a site, call `list_recipes` → pick one → `get_recipe` → follow the steps using
  `click`/`fill` (each still approval-gated at Act). The recipe is *guidance*, not an auto-runner.
- **Follow the intent, not just the literal rows.** A recipe's `description` may describe a
  *generalizable pattern* (e.g. "the statements are in `dl.dl-row`; click each row's download
  button, incrementing the row index until none match — there may be more rows than were
  recorded"). When the live page has more items than the recorded steps, read the page and apply
  the pattern to **all** of them rather than replaying only the captured rows.

---

## 6. How a call resolves

Every invocation returns an **InvokeResult**:

```ts
// success
{ ok: true, output: <tool output> }
// denial / failure
{ ok: false, reason: <DenyReason>, message?: string }
```

### Deny reasons (all fail closed)
| `reason` | Meaning | What you should do |
|----------|---------|--------------------|
| `permission_denied` | Tab mode is below the tool's `minMode` (incl. AI Off) | Tell the human which mode you need; don't retry blindly |
| `invalid_input` | Input failed the tool's schema | Fix the input shape and retry |
| `approval_required` | Needed approval, none was granted (incl. card **timeout ~120s**) | Re-request only if the human still wants it |
| `approval_rejected` | The human pressed **Reject** | Stop. Do not retry the same action |
| `revoked` | Grant was pulled mid-flight ("Stop AI" / mode change / tab closed) | Treat access as gone; re-confirm before continuing |
| `timeout` | Handler exceeded its time budget | Retry once; if it persists, report it |
| `handler_error` | The tool threw while executing | Report the `message`; the page may be in an odd state |
| `unknown_tool` | No such tool | Check the name against `safecobrowser tools` |

### Approval flow (for `click`, `fill`, `run_js`)
1. You invoke the tool.
2. SafeCoBrowser shows the human an **approval card** with the concrete effect (selector, or the full
   `run_js` script). Your call **blocks** meanwhile.
3. The human **Approves** → the tool runs and you get `{ ok: true, ... }`.
   The human **Rejects** → `approval_rejected`. No answer within ~120s → `approval_required`.
4. The human may have **auto-approve** on for that tab (one toggle for actions = `click`/`fill`,
   a separate one for `run_js`). Then approved calls run **without a card** — but are still fully
   logged. You can't tell the difference from the result, and you shouldn't rely on it being on.

---

## 7. Sessions, epochs, and instant revoke

- There is one authoritative session state per tab, carrying an **epoch**.
- Tool handlers and approvals re-check the **live epoch at execution time**. So even after a card
  is approved, if the human hits **Stop AI** before the click actually fires, the action does
  **not** run (`revoked`). This is what makes "instant revoke" real.
- **Practical consequence:** never treat a successful grant as durable. If you get `revoked` or a
  sudden `permission_denied`, the human pulled access — pause and re-confirm, don't hammer.

---

## 8. Multi-tab behavior

- SafeCoBrowser can have several tabs, each with its **own** container (isolated cookies/logins) and
  its **own** permission mode.
- **You always operate on the active tab** (`safecobrowser status` → `tab`). You cannot switch tabs
  or address a specific one — the human drives tab focus.
- The grant **persists across navigation** within a tab (changing URL does not drop your access);
  only Stop AI / a mode change / closing the tab clears it.

---

## 9. The privacy filter (you may see redacted text)

The human can turn on a **privacy filter** — their own list of sensitive strings (name, address,
account numbers) that SafeCoBrowser replaces with labels like `[Name]` or `[Account]`. There is **no
tool** for it and you can't see or change it; like the permission mode, it's the human's control.

What it means for you:

- **What you read may be redacted.** `read_page`, `inspect_element`, `screenshot`, and even
  `run_js` all read the **rendered DOM**, which is exactly where redaction happens — so you'll see
  `[Name]` where the real value was. This is intentional: the human is hiding it from you (and from
  anyone watching their screen). Treat the label as a value you're not meant to have.
- **Don't try to defeat it.** Do not probe for the hidden value — no scraping attributes, input
  `.value`, network bodies, or storage to reconstruct it. If you genuinely need it to finish the
  task, **ask the human**; they can fill the field for you or turn the filter off.
- **It's best-effort, not total.** Redaction covers **visible text nodes only**. It does **not**
  redact form-field values, element attributes, URLs in `read_network` metadata, or text that's
  reformatted/abbreviated/split across elements — so you may still encounter a real value there. If
  you do, treat it as sensitive: use it only for the task at hand and don't echo it back into the
  conversation or anywhere it isn't needed.

---

## 10. Workflow patterns that work

- **Look before you leap.** In Inspect/Act, call `read_page` (and `inspect_element`) to find the
  *real* selectors before `click`/`fill`. Don't guess selectors blindly.
- **Diagnose with Inspect.** For "why did this fail?", combine `read_console` + `read_network`
  (look for non-2xx statuses) before proposing a fix.
- **Fill then submit, deliberately.** `fill` never submits. After filling, request the submit as
  its own `click` so the human approves the consequential step explicitly.
- **Re-read after navigation.** If a `click` navigates the page, call `read_page` again — your
  prior DOM knowledge is stale.
- **Screenshots for visual checks**, saved to a file (the base64 is big).
- **Respect denials as signal.** `approval_rejected` means "no" — pick a different approach, don't
  re-ask the same thing. `permission_denied` means "raise the mode," which only the human can do.
- **Minimize `run_js`.** Prefer Read/Inspect/Act tools; they're lower-risk, clearer in the audit
  log, and less likely to be rejected. Use `run_js` only when nothing narrower works, and keep the
  script tight and readable (the human reads it in the card).

---

## 11. What you must NOT do / cannot do

- ❌ Change the permission mode, or look for a backdoor to. (Only the human's UI can.)
- ❌ Access cookies/passwords/the profile directory, or other sites' sessions.
- ❌ See blind-period content (anything before Read was granted on this tab).
- ❌ Act on a background/other tab — only the active one.
- ❌ Assume a grant lasts. It can be revoked between two of your calls.
- ❌ Treat `fill` as submit, or chain a hidden submit into another action — submitting is its own
  approved `click`.
- ❌ Try to recover values hidden by the human's privacy filter (§9).

---

## 12. The audit log

- Every brokered decision (allowed **and** denied) is appended to
  `~/.safecobrowser/audit-log.jsonl`, hash-chained for tamper-evidence, surviving restart.
- `click`/`inspect_element` log their **selector**; `run_js` logs the **full script**; `fill`
  logs the **selector but never the value**.
- The human can review it live in the in-app **Activity** panel, or via `safecobrowser audit` /
  `safecobrowser audit verify`.
- Implication for you: be transparent and minimal. Anything you do is attributable.

---

## 13. Cheat sheet

```bash
# Is SafeCoBrowser up? which tab is active?
safecobrowser status

# What can I call right now?
safecobrowser tools

# Read tier
safecobrowser invoke read_page
safecobrowser invoke screenshot

# Inspect tier
safecobrowser invoke inspect_element '{"selector":"form button[type=\"submit\"]"}'
safecobrowser invoke read_console '{"limit":50}'
safecobrowser invoke read_network '{"limit":50}'

# Act tier (each prompts the human unless auto-approve is on)
safecobrowser invoke click '{"selector":"a[href*=\"/contact\"]"}'
safecobrowser invoke fill  '{"selector":"#email","value":"hi@example.com"}'

# Develop tier (always shows the script to the human)
safecobrowser invoke run_js '{"script":"document.title"}'

# Audit
safecobrowser audit 20
safecobrowser audit verify
```

**Result shape:** `{ ok: true, output: … }` or `{ ok: false, reason, message? }`.
**If denied:** consult §6, say what you need, and let the human decide. You are a guest with a
key the host can revoke at any moment — work that way.
