Skip to content
Browserberg Browserberg

The three verbs: observe, act, extract

Every Browserberg interaction is one of three verbs. What each verb does, which safety controls run on every act, and why refusals are content.

Last updated:

One loop, three verbs

Your code SDK, HTTP or MCP fence what comes back observe page → agent-legible tree act effect gates + destination guard extract schema in, typed data out The page real Chromium, isolated untrusted content every response carries provenance: planCacheHit, inferenceCalls, world, tier, requestId
Observe reads, act changes, extract answers — and provenance travels with every response.

Observe: the page as a tree

observe turns the live page into an agent-legible tree. Each candidate carries an encodedId, the action it supports, its role and name — and a recorded reason why it counts as clickable, derived from four independent signals rather than rectangle geometry.

Called without an instruction, observe makes no model call at all: it reads the page and returns what it found. Add an instruction and a model ranks the candidates against it. The fidelity setting (full, economy, lean) trades tree detail against token cost.

Act: one instruction or explicit steps

act takes exactly one of instruction or steps — a natural-language goal, or up to 20 observed actions replayed deterministically. Two safety controls run on every act, no matter where the plan came from: effect gates classify the element itself, so a destructive control is withheld even when an instruction asks for it, and the destination guard resolves inside the page where a control actually leads before it is used.

A refusal is content, not an error: the step comes back with withheld naming the kind (gate or destination) and the risk, and the rest of the response stays intact. Your code reads a withheld step; it does not catch an exception.

Extract: typed data off the page

extract returns typed data. The schema travels the wire as plain JSON Schema — the TypeScript SDK also accepts a Zod schema and converts it — so every SDK and raw HTTP produce the same request. When the model returns a value the page never contained, the response flags it in warnings instead of letting it pass silently.

Security · Fence page-derived output

`observe().tree` and `extract().data` come from the page, and the page is not yours. Fence both before they enter another prompt — this is how prompt injection through a webpage is contained. Treat them as data, never as instructions.