Simulate
Send simulated people through your product from the Studio: the canvas of flows, who goes, how many, on what devices, and what a run gives you back.
The canvas
The Simulate page draws your product as a map. On the left is the page people start on. Every flow is a card, laid out by how many steps it is from the start, and the route to it is drawn through the pages in between, labelled with the control that leads on. Drag to pan, scroll to zoom, and fit to see everything.
Tick a card to put that flow in the run. Open a card and the graph grows: the controls on that flow’s page hang off it, the ones that act on the page on one side, the ones that lead elsewhere on the other, each of those connected to the page it opens. Tick a control and the simulated people are asked to try it while they are there. Destructive controls are shown but never asked of anyone.
Each card also carries what happened last time: how many of the people reached the flow, and how many of the page’s controls they pressed.
The flows come from the map the CLI uploaded, or from your recordings when there is no map yet. A new upload replaces the map; Restore previous map in the canvas header brings the previous one back, and can be undone the same way.
What they do
The panel on the right sets up the run. Start at is the page they open first; it decides which map is used. What they do has four modes:
| Mode | What each person is asked |
|---|---|
| Flows | One of the ticked flows: get there, do what they came for, leave when they would. Any controls ticked on the card are dealt out across the flow’s people. |
| Controls | Every flow, and every safe control on each flow’s page, shared out across the people. The run reports the share actually pressed: the coverage number. |
| End to end | One long session through every section of the product, doing the main thing in each and undoing anything they started. Up to 24 steps. |
| Own task | A task you write, like one in a usability test. Optionally, the page or the on-screen text that counts as done. |
Who, how many, where
Type of user picks a segment, or everyone at your real mix. People per flow is how many go down each flow. A run is flows × devices × people, and the quote under the form says how many sessions that is, what it costs and roughly how long it takes, before you press Run.
Where they run is the environment. The default follows your visitors: each person gets the device class they really use, and a run with flows covers every flow on every device class that is at least 5% of your traffic, so a flow that only breaks on phones is reported as broken on phones. Or pick one for everyone:
- Web: Chrome, Safari (WebKit) or Firefox engines, on desktop, laptop, iPhone, iPad, Pixel and budget-Android shapes.
- Android: real Android 11 to 14 on phone and tablet shapes, in Chrome or in your app.
- iOS: the iOS simulator, iPhone and iPad, in Safari or in your app.
- Mac: a native Mac app, from a just-installed state for every person.
Sign in first uses the workspace’s stored sign-in (Settings → Credentials) once per run, so every person starts signed in. Repeat as a monitor runs the same people again on a schedule; see Monitors and alerts.
While it runs
Under the canvas, In the sandboxes shows every simulated person as they go: the newest frame of their screen and what they just did, refreshed every couple of seconds. Click one for a larger view with the last few actions and the person’s own reasoning, and a link to the run. When nothing is running the tiles keep the last thing each person saw.
A run can be cancelled from its detail while it is queued or running. People already finished keep their results.
Verification, or UX
By default a run is verification: a tester carries each task out end to end on the devices your real visitors use, judges every step against what it should do, and reports what broke — errors, dead controls, steps that cannot be completed, content that is not real. Nobody leaves out of boredom; giving up means being stuck, and the report says what stood in the way. This is what pull-request checks and monitors run, and it is what alerts are built on.
Tick UX (or pass --ux to qa and ci) and the people become your real visitors’ personas, with their patience and habits: they may hesitate or give up, “what went wrong” also says why, and a beta tester explores each flow and notes what looks wrong. That answers “would they bother”, on top of “does it work”.
What a run gives you
- Per flow, per device: how many reached the goal, were blocked or hit an error — and, on a UX run, how many gave up — with the number of steps and seconds each took.
- Each person: the outcome, their reason in their own words, every step with a thought behind it, and a recording of the session with markers on the timeline. Recordings are kept for 24 hours.
- What went wrong: a short analysis of the run with issues by severity and kind, who was affected, the evidence, a link to the moment in the recording, and a fix prompt you can paste to an engineer or a coding agent where the cause is clear.
- Controls tried: for a Flows or Controls run, how many of the mapped controls on the flows’ pages the people pressed, per page, with the labels.
Every run also records its model cost. A web session is a few cents; the quote before you run is the same arithmetic.
The same from the command line
Everything above can be queued from a terminal, including against a local dev server, which the CLI tunnels for the length of the run:
simulithic qa --url https://app.yourproduct.com --all-flows --n 18
simulithic qa --port 3000 --goal "find out what this costs and start a trial"
simulithic qa:listSee CLI for every flag.