Agents & MCP

Why the loop is the product

A conventional agent driving an app does this: screenshot, infer, tap, screenshot, infer, tap. Every action costs an inference, every screenshot costs thousands of tokens, and any of them can hallucinate a button that is not there. Twenty steps is twenty chances to go wrong and twenty times the bill.

Squiggle makes the model a compiler. It emits one goal-conditioned intent program — a plan with branches, declared escalation points, and a goal that gets verified afterwards — and the runtime executes it against the live semantic graph. Turns become O(escalations) rather than O(actions).

intent: check out with a saved cardgoal:  route: OrderConfirmation  visible: { text: "Order placed" }plan:  - tap: { testID: cart }  - scrollUntilVisible: { element: { testID: checkout }, direction: down }  - tap: { testID: checkout }  - branch:      if: { visible: { text: "Add a card" } }      then:        - escalate: { reason: "no saved card on this account", want: decision }      else:        - tap: { testID: pay-now }  - waitFor: { route: OrderConfirmation, timeout: auto }

On the reference journey, run from a dirty app, that is 3/3 passes at zero model turns, and 20/20 on repeat. The model was called once, to write the program.

What happens before anything is tapped

A program is gated offline before it is allowed to touch the device, through a pyramid that degrades honestly rather than silently:

Static

Does every selector name something the route graph has actually seen? A program aiming at a screen that does not exist is stopped here, with zero live actions.

Replay

Do the program’s assertions hold against the newest relevant recording? This checks the plan against territory we have already been through.

No coverage

A program with no covering recording degrades to static-only with a note saying so. It never passes silently — an ungated run that claims to be gated is worse than no gate.

alternatives turns this into best-of-N: emit several candidate programs in the same turn, let the gates score them all, run the winner, and keep the runners-up as pre-validated fallbacks. Zero extra model turns.

The deny rail

An actuation blocklist enforced at delivery time — below the model, in the interpreter, not in a prompt. A model cannot talk its way past it because it is not being asked.

Your app can declare what a control is, and a declaration always beats an inference from copy:

<Pressable  testID="delete-account"  {...agentHint({ destructive: true, role: 'button', note: 'permanent, no undo' })}/>

A rail hit that the run genuinely needs escalates as needs_approval. The grant is scoped, single-use and logged — and approvals are never cached.

Escalation packets

When the runtime hits a real decision point it freezes and builds a packet: the program counter, the reason, delta narration since the program started, ranked heal candidates, the log tail, and an evidence coordinate. Big payloads stay behind resource references rather than being pasted in.

A seeded rename — someone changed a testID — is repaired by exactly one escalation in a 1,143-byte packet. The answer is one constrained arm: pick, patch, retry, skip, abort, approve, or learnInterrupt.

Why it gets cheaper the more you use it

This is the part that compounds, and it is worth reading as a list because each item removes a future model call permanently:

Delta narration — 96% cheaper perception

After the first look, sinceLastLook returns what changed rather than a second full dump — measured at 96% cheaper while reconstructing ground truth exactly. Every subsequent observation in a session is nearly free.

Learned interrupts — one escalation per popup, ever

An unhandled interstitial escalates once. learnInterrupt persists the handler, and interstitial autopilot runs it below every step boundary from then on. The tenth time that cookie banner appears it costs nothing.

The flow library — flows are skills

A successful program can be saved, generalised into a parameterised version, and replayed. Replaying a library flow costs zero model turns. Your agent’s capability grows as a library rather than as a longer prompt.

The route graph — knowledge instead of exploration

Every run records observed transitions with counts and p95 timings. predict answers “what has tapping this historically led to” with no live actions; plan gives the shortest known path to a screen; and timeout: auto takes its number from what the app has actually done rather than from the model guessing milliseconds. Unknown transitions return “no history” — the graph never guesses.

The decision cache — a repeat escalation costs nothing

Answers are memoised by screen signature, failure kind, goal shape and app build. A hit costs zero model calls and still flows through the rail and the gates, so a signature collision produces a failed gate rather than a wrong action.

Inferred contracts — tests nobody wrote

What held across every green recording of a screen becomes a contract, with its confidence attached. Violations report as contract_drift beside the goal, never as a hard failure, because inferred things can be wrong.

The tools

Eighty tools over one persistent warm session, held across calls, so each verb is a ~3 ms socket round trip rather than a process boot. Registering the server never boots a simulator.

Read this first: status

The capability matrix — transport, session, injected?, dylib freshness, captures, and a map of which capabilities are unusable right now and why. An unknown dylib version is treated as stale, never as trustworthy.

Perception

describe (interactive-first, token-budgeted, stable short refs like n42, or delta narration), find (a confidence verdict — unique / ambiguous / none — with ranked alternatives up front, so you disambiguate before acting rather than after failing), query, assert, wait_for, and screenshot as the escape hatch, never the default.

Frames

filmstrip, for the defects the graph cannot see — because the graph is correct the whole time and only the rendering is briefly wrong. An overlay painting in the corner for one frame, a row drawn with new content in its old position, a janky drag: describe reports a correct screen at every point and assert passes. It renders a labelled contact sheet from a recording — each frame stamped with its timestamp and the screen route it was on, and old→new on the frames inside a navigation, because a transition shows two screens at once and either name alone would be half wrong. A second mode returns per-frame brightness over a crop, which is the only way to prove a single-frame artifact: a sheet samples every ~45 ms at best, so a clean strip is not proof that nothing flashed. Needs ffmpeg; optional, and nothing else depends on it.

Actuation

tap, input_text, swipe, scroll, back, launch_app, run_flow. Each takes a ref from your last look as well as a selector. A success means dispatched, not effected — which is why programs are the fast path. Each of them settles and returns the screen delta as observed in the same reply, so an action does not have to be followed by a look; observe: false gets the bare ack.

Reaching what is off screen

scroll_until_visible loops server-side and costs one round trip where scrolling and looking by hand costs four. Read its found — ok only says the loop ran.

Batching

batch runs several of those tools in one round trip — same names, same arguments, nothing new to learn — stopping at the first failure and observing only at the end. It is the rung between a single poke and a program, and deliberately weaker than one: no offline gate-check, no escalation, no loops, no goal:. Those absences are the reason to graduate.

Programs

run_intent’s program argument is typed inline with the common subset, so a first program needs no schema fetch; the shape is permissive, and unlisted step kinds and selector fields pass through rather than being stripped. intent_schema remains the full grammar. intent_wait (long-poll to the next boundary), resume_intent, intent_stop, and steer — a hint injected into a live run, enforced at the guard seam rather than offered as advice, because a hint the program can ignore is a suggestion.

Memory and planning

library_list/library_save, predict, route, interface (the route graph compiled into a typed per-screen SDK), diff_runs, affected_by_diff (which library flows an interface change puts at risk — expanded through the import graph, so a change to a shared hook marks the screens it reaches, not nothing).

Evidence

explain walks the deterministic why-chain behind a coordinate — blame is a query, not an inference — and verify_citations re-resolves every evidence link in a narrative and flags the ones that do not hold. An uncited report is reported as uncheckable.

Data

The whole perf and product surface — the MCP catalogue is a superset of those CLI families, wrapping the same in-process functions rather than shelling out.

Serving fewer of them

The full catalogue is 80 tools — roughly 27,400 tokens of schema in every conversation, whether the session drives an app or profiles one. Set SQUIGGLE_MCP_MODE on the server and it registers fewer: drive drops the perf and product families (51 tools, about 7,400 tokens back), analyze drops the device half instead, and qa / perf / a11y / explore serve exactly the 13–17 tools their subagent runs under. Unset serves everything.

A scoped-out tool is absent, never disabled. That distinction is load-bearing: a disabled tool is refused by the protocol layer before any of our code runs, so the agent gets a generic error instead of an explanation. An unrecognised mode value serves everything and says so — hiding tools because someone mistyped an env var looks exactly like a broken product.

What a client shows you

Every tool carries a derived display title, stamped from the tool name in one place so none of the eighty can forget — a client’s UI and its permission prompts read “Launch app” where the wire carries launch_app.

Tool icons and in-flight progress were built here and removed, which is worth recording because they are the obvious next reach. Icons are priced by transport, not by preference: an HTTP server may serve images from its own authority, a stdio server may use file:///, and any transport may use data:. Squiggle is stdio, so the cheap https:// option is nobody’s authority and renders nothing, and the schemes that do work cost 2,900–5,100 tokens across the catalogue — against the 7,400 that mode scoping reclaims. Decoration at half the saving is the wrong trade.

Progress is narrower than it sounds. A server can emit notifications/progress while a call runs, and only if the client sent a progressToken. There is no completion notification and no checkmark primitive: the tool result is the completion signal, and what a client draws when a call finishes is its own rendering of that result. intent_wait does stream progress, because there the payload is a real program counter rather than an elapsed-time label.

What each tool costs you

Every tool carries a licence posture, and the four classes are the whole taxonomy: 1 free, 24 drive, 32 analyze, 23 agent.

free — status

Exactly one, and it is the one that tells you why the others are refusing. A capability report that needed a licence to read would be the worst possible thing to gate.

drive

Anything that touches the device: perception, actuation, recording, the flow library, branch, crawl.

analyze

The perf and product families, plus query, explain, diff_runs and affected_by_diff — everything that reads the store.

agent

The intent runtime itself: programs, escalations, approvals, briefings, probe_read, interface, invariants, legibility.

The mapping is not a comment — it is a table three test suites enumerate, one of them against a live server, so a newly added tool cannot quietly inherit a default. It fails closed: unclassified means paid, and a fail-closed default is safe but is still not a decision, which is what the suite exists to say.

Read code-mode

probe_read collapses roughly twenty-five flat read tools into one verb. You write an async function body against a typed read capability surface and return only what you need — filtering and aggregating server-side:

// probe_read — write a program, return only what you needconst slow = await perf.sql(`  select name, avg(dur_us) avg from spans  where kind = 1 group by name order by avg desc limit 5`);const insights = await perf.insights();return { slow, blocking: insights.filter(i => i.severity === 'error').length };

Returning 20 MB so you can grep it in context is the exact failure this exists to prevent, and there is a hard result cap. Reads are low-consequence, so it trades “zero parse failures” for a tight runtime-error → edit → rerun loop.

Acts get no such treatment. They stay the governed intent DSL, because a tap can log out, delete, or pay. Security is layered and the sandbox is the last layer, not the guarantee: actuation is absent rather than gated (there is no drive verb to reach for), the authorization proxy runs in the parent, the database rejects writes, there is no network or filesystem, and every run is traced in a killable child process.

Scoring how well it drives

Everything above is a claim about a loop. squiggle benchmark is where the claim gets a number: a versioned suite of mobile-UI journeys, each one a natural-language objective handed to an agent, and the score is whether the task’s assertion held afterwards. That is the only reason this can exist at all — grading a mobile-UI agent is normally a human reading screenshots, and Squiggle grades from the runtime graph instead, so a result is auditable and re-running a recorded one gives the same number.

$ squiggle benchmark list                     # the task set$ squiggle benchmark run --reference          # the oracle, 0 model turns$ squiggle benchmark run --agent 'claude -p "$SQUIGGLE_BENCHMARK_PROMPT"'$ squiggle benchmark history 20               # the recorded trend

--reference runs each task’s own oracle program at zero model turns — the CI-shaped form, and a regression gate on the suite rather than on any agent. --agent <cmd> runs your command once per task with the prompt in an environment variable, waits for it to exit, and scores the end state through the same path. Nothing supervises how it drives, because the point is to score any MCP-speaking agent and not only the ones that use Squiggle the way we would.

Three rules do the work, and each of them refuses something:

The task owns the goal

Whatever goal: a submitted program declares is discarded and the task’s substituted. Otherwise a program passes by asking for something easier than it was set, and the leaderboard measures ambition.

Every task carries a counter-example that must fail

Half of a gate is proving it is not always-green. A goal that both the oracle and the deliberately-wrong program satisfy is reported indiscriminate and scores nothing — its passes mean nothing either.

unproven is neither a pass nor a fail

No device, no app, or a gate refusal before actuation all mean no data. Those tasks leave the denominator and are counted beside it, so a score over three of fifteen tasks can never be quoted as a score over fifteen.

Setting it up

$ squiggle init --mcp                # writes .mcp.json for you (merges, never overwrites)# or by hand, either way:$ claude mcp add squiggle -- squiggle mcp# .mcp.json at your project root:$ { "mcpServers": { "squiggle": { "command": "squiggle", "args": ["mcp"] } } }

squiggle init --mcp is a merge, not a write: every server already in your .mcp.json stays, along with any keys your client puts beside them. If squiggle is already registered it changes nothing — a command you pinned yourself is a command you meant. It is opt-in for the same reason: that file configures your agent client, not us.

Then install the methodology that travels with the tools:

$ squiggle agent kit --install          # install skills, subagents and commands into .claude/$ squiggle agent kit -o ./kit        # or stage all of it, hooks included, to read first$ squiggle agent rules               # an AGENTS.md fragment, generated from the catalogue

The kit is a build artifact, compiled from the DSL schema, the capability library and the mode briefings. It ships four skills (squiggle-setup, squiggle-drive, squiggle-perf, squiggle-triage), four subagents (qa, perf, a11y, explore), slash commands, and optional hooks. Every skill is also readable over the wire as a squiggle://skill/… resource, so a client that is not Claude Code still gets the guidance rather than eighty tools and no instructions.

--install writes the seventeen files a harness loads and withholds the hooks, because merging those edits your settings.json and arms a bench-on-stop that runs a real device replay — not something an install should do behind your back. -o <dir> stages all twenty-one so you can read the hooks before arming them. A destination that is something else — a symlinked .claude/agents, say — is reported and skipped rather than overwritten, and everything else still installs.

The kit tells you when it is behind. Each skill carries the kit’s own content digest in its frontmatter, and both squiggle doctor and the MCP server compare it against the build you are running — the server by attaching a one-time note to a tool result, which is the only channel your agent actually reads. It is a hash of the generated content rather than the product version, so an upgrade that did not change the kit asks you to do nothing. When it does say something, run squiggle agent kit --install again.

A mode is a prompt, a goal template and a hard tool allowlist — not emphasis. The allowlist becomes the generated subagent’s frontmatter, so a perf agent cannot reach a drive verb by accident.

Governance

One live run per device

Agent and human can never actuate the same simulator at once.

Budgets

Escalations, actions and wall-clock, enforced by the runtime and reported in the run. A ceiling you hit at speed is a wall, so the runtime can advise “finish fast” and let the answerer drop optional work instead of losing the run.

Every acting call is logged

Each acting tool call appends to the project’s shared history tagged source: mcp, the same file the CLI and the desktop write. A human reading the log sees both halves of the work.

Every intent run auto-arms a recording

A failure already has its reproduction.

Licensing refuses inside the protocol

The server always starts — a non-zero exit reads to a client as a broken install, so the customer files a bug instead of buying a licence. It serves, and the refusal travels in the result envelope for the agent to relay. There is deliberately no activate tool: that would mean a licence key typed into a conversation, and into whatever that transcript syncs to.

A human can answer instead

The Agent Center in the desktop app watches a run’s journal stream and lets you answer its escalation packets by hand. Once a packet is structured, a human is a perfectly good model — and free. The same seam serves Claude Code over MCP, a scripted answerer in a test, or you.