Devices & Studio

Bring-up is one click

Selecting a simulator drives a single de-duplicated bring-up sequence: boot it, launch the configured bundle if it is already installed, else resolve or build a simulator .app, install it, launch it, and open its dev client against Metro. You do not run five commands in the right order; you pick a device.

The whole sequence is a tracked process, so it reports itself in three places at once — a per-phase spinner in the device row (Booting, Installing app, Building app, Launching app, Opening app), the header status rail, and the notification centre. Device discovery stays live, so a simulator booted from Xcode appears without a refresh.

The live frame

Frames come from the simulator’s framebuffer surface zero-copy — no simctl spawn, no PNG encode/decode round trip. The desktop makes one copy into a buffer and draws it in a GPU canvas. Capture is seeded by the device rather than polled, so an idle simulator costs nothing, and it downscales natively on the way out so memory stays bounded.

The frame is drawn on the UI thread with no React commit per frame, which is why it stays smooth while the rest of the app is doing work. Streams are pooled per device, so several simulators can stream at once — which is what the multi-device grid is: expand the panel and every booted device becomes its own interactive cell, each with its own graph session keyed by the udid the app announces.

Driving it

The Interact toggle arms input. Everything then goes through the native input engine as real HID:

Tap and live-streamed drag

A drag streams down/move/up as it happens, with moves coalesced so a 60 Hz drag cannot flood the queue, and a guaranteed lift on cancel so the digitizer never sticks.

Home and Back edge gestures

A drag starting in an edge band is classified and tagged with the matching digitizer edge bit, so the simulator recognises it as a genuine system gesture while still following your finger.

Wheel and trackpad scroll

Scrolling over the canvas plants a virtual finger that follows every delta and lifts after a quiet period, re-planting near an edge so a long scroll does not trip the edge-gesture bands.

Mac keyboard capture

While interactive, your keystrokes route to the simulator — named keys and

⌘

/

⌃

combos by key code, printable characters through the injected agent’s caret insert (one ~3 ms call) with per-char HID as the fallback.

⌘ C

and

⌘ V

bridge the simulator pasteboard both ways. System shortcuts still reach macOS.

Hardware buttons

Home, Lock and volume, from the toolbar.

Device skin

A procedural bezel derived from the device family, with rounded-corner screen clipping and pressable edge buttons. Sizes are fractions of the screen’s short edge, so it scales with the window.

Overlay modes

The overlay cycle draws what the runtime knows on top of what the device is showing. Cycle through: none → renders → all → accessibility → heatmap.

⊘ No overlay

A clean, unannotated device — for demos and for screenshots.

Renders

Component bounds tinted by how often each has re-rendered, so a component that is repainting on every keystroke is visible as a colour, not as a number in a list.

All

Every node the graph resolved a rect for.

♿ AX tree

The device’s live native accessibility tree — bounds and labels, mapped onto the frame. When the app is injected this is push-driven: it repaints only when the tree actually changes, and is silent while idle. Uninjected it falls back to self-paced polling of an external sweep. This is also the plane your flow selectors fall through to, so an element missing here is an element a flow cannot name.

🔥 Heatmap

Every touch as a density blob plus every swipe as a vector — see Interaction analytics.

🪟 Native overlay

A second, click-through window parked over the real Simulator.app window, drawing the same rects with Core Animation. For when you want to work in the actual simulator and still see the annotations. It hides itself when another window is stacked over the simulator.

Runs are bounded windows

A run is a capture window, not a connection. It gets its own id and its own row in the store, so one continuous app session can hold several discrete runs — start → interact → stop → inspect. A run auto-starts on connect and on relaunch; ● Start run mints a fresh one, ■ Stop run flushes and freezes it on screen, and Reset run wipes the current one in place while continuing to capture. Past runs are selectable from a picker across the analytics, performance and product screens, and deletable individually.

This is the same shape a CLI invocation gets for free: one squiggle flow is one run.

One project at a time

Every store plane — flows, recordings, perf.db, history, screenshots, agent memory — derives from a single active project root, chosen explicitly through a real folder picker and switchable from the title bar or the home banner. Switching remounts every project-scoped screen, so no data survives the change. It is the same .squiggle the CLI resolves by walking up from your working directory, which is what lets a flow you ran in a terminal show up in Studio’s history.

Flow Studio

Flow Studio turns the same canvas into an authoring tool, driving the same engine the CLI does — so a flow authored here runs identically in CI.

Record

It captures both desktop-mediated taps and taps you make directly in the Simulator window, resolving each one against the live accessibility tree and the graph into a semantic step. Generic container labels and decorative roles are skipped. Composite labels are split: iOS joins a nav bar’s children into one accessibility label — "Forward, Animations" — and recording the whole string would produce a selector that matches nothing, so the recorder takes the segment that is actually the title.

Edit

A visual step list — add, remove, reorder, edit — with a grouped step palette (Interact, Wait/Assert, Control flow, Device), so a whole flow can be hand-built rather than only recorded. Delete, revert and save live per step as well as per flow.

Organise

Flows group into folders — real subdirectories of .squiggle/flows/, so the structure is in the repository rather than in a database. Sort by name, last replayed or date added, and move a flow between folders by dragging it.

Replay, with eight toggles

The replay log streams each step with a spinner on the live one, then freezes as the last run with its verdict and timings. Five of the checkboxes under Replay options are the delivery knobs described in Speed & delivery, under the same names as their CLI flags — In-process delivery (on by default), Disable animations, Direct scroll, Fast settle and Speculative resolve. The other three change what a run does rather than how fast it does it: Auto-correct selectors runs the flow’s prerequisite as a pre-phase, so the app starts from a known state and the correction is reported outside steps; Capture network installs the pass-through NSURLProtocol; and Continue on failure reports every step instead of stopping at the first red one.

Doctor and Crawl, in place

Both reuse the live connection rather than opening a second server: Doctor runs the connectivity and graph-health check, Crawl runs the bounded autonomous crawler and loads the flow it discovered. Their reports share the replay panel.

The relay, so the terminal still works

Whenever an app is connected the host automatically re-serves that session to CLI consumers on a second port. There is no toggle. So squiggle flow runs against an app Studio is already holding rather than fighting it for the socket — the CLI dials the relay when the primary port is taken.

While a CLI is attached, a CLI flow attached chip appears and the desktop’s own Replay button is disabled, because two drivers on one app is a race, not a feature.

The rest of the app

Home

Devices, the connection status, the active project banner, and whatever is currently running. The place a session starts.

History

Every run this project has produced — flows, benches, crawls, recordings — with its outcome and duration, and a re-run that replays the exact invocation recorded with it. Shared with squiggle history, because both read one JSONL file.

Agent Center

The intent runtime with a face on it: programs, their escalations, the decision cache, the flow library and the approval rail. See Agents & MCP.

Settings

⌘ ,

Appearance, sound, the licence, and Install CLI in PATH…, which symlinks the bundled squiggle binary into ~/.local/bin.