How to test an iOS app with an AI agent

A model is good at the part of an end-to-end test that breaks most often: getting through the flow. “Complete onboarding, don’t allow notifications, reach the paywall” survives a renamed button and a new screen in the middle. A model is a poor judge of its own run. An agent that says “done” is reporting what it meant to do.

mobster verify gives each job to the right worker. A model may drive. The verdict comes from assertions you wrote before the run, checked in code against the app’s accessibility tree. In the words of the Checks reference: what the model says about its own run is never an input to the verdict.

That’s the difference from AI assertions that ask a model. Maestro’s assertWithAI, for one, uploads a screenshot to an LLM with your assertion and takes back true or false, and Maestro marks it experimental (its docs, read 8 October 2026). A Mobster verdict is a fact about the screen, with the frame that shows it.

An assertion is a fact about the screen

An assertion is a small object with one kind key. Mobster checks it against the tree WebDriverAgent reports for the app, with no model, OCR or pixel comparison.

A selector matches on id (the accessibility identifier), label, value, role, enabled and selected, and every field you give must match. label and value take an exact string or a /regex/.

A check file puts the flow and the assertions together. This one is written for the paywall of Daybreak, the sample app in the CLI’s repository:

version: 1name: A new user sees three plans on the paywallapp:  bundle: dev.mobster.daybreak  path: .mobster/build/Build/Products/Debug-iphonesimulator/Daybreak.appsteps:  - Complete onboarding as a new user. Don't allow notifications.  - Reach the paywall.expect:  - text: Choose your plan  - count: { id: /^plan_/ }    equals: 3  - value: { id: plan_annual }    equals: $39.99 / year  - visible: { label: Restore Purchases, role: button }  - no_text: Loadingbudget: { max_seconds: 180, max_usd: 0.25 }

An unknown key at any level is an error, so a typo can’t quietly weaken a check. Text is normalized before it’s compared (curly quotes, dashes and invisible marks), so a label that says Don’t Allow matches Don't Allow. text and no_text never join two elements: “$39.99” in one and ”/ year” in the next don’t make “$39.99 / year”.

One settled screen decides

A flaky verdict usually comes from reading a screen mid-animation. Mobster decides on a settled screen, with every assertion on the same read:

1. It reads the tree, waits 0.3 s and reads again. Two reads with the same fingerprint are settled.
2. When a settled read satisfies every assertion, that read decides.
3. Otherwise it reads again every 0.5 s until the timeout, 5 s by default (--assert-timeout, 0 to 60).
4. At the timeout the last read decides, and the result says whether the screen was still changing.

So a no_text: Loading that holds on one read and a text: Plans that holds on another never combine into a pass. Content behind a full-screen cover or scrolled out of view doesn’t count as shown, and an agent scrolls to it with swipe and until, which stops as soon as an assertion holds.

Four verdicts, and two rules that keep a pass honest

The exit code is the verdict, so a script can branch on it:

A crash fails the run as app_not_running, whatever the assertions say. Then two rules stop a pass that proves nothing:

• No assertions ends needs_review. There is nothing to decide by.
• Held before the flow ends needs_review too. Right after launch, before any step, Mobster takes one read. If every assertion already holds on it, the check doesn’t show that the flow works. Without this rule, a paywall check that only asserts “Choose your plan” would pass with onboarding broken, whenever the app happens to open on the paywall.

Three ways to drive, one way to judge

Launch-only and key-less runs send nothing anywhere. In the key-less mode, your agent’s model drives and Mobster only judges: Give Claude Code a real iPhone and Test your iOS app with Codex walk through that loop. Checks run on simulators Mobster manages, and mobster verify --device checks an app you already installed on your USB iPhone without clearing its data.

The proof comes with the verdict

Every run writes a folder with result.json, report.html, the run’s check.yaml, its events and its frames. The report is one HTML file with no external requests. It shows each assertion with what was actually on screen, each step with its frame, and the verdict frame beside the accessibility overlay: every element Mobster read outlined, and each asserted element in green when it held or red when it failed.

A failure says what it found instead. Daybreak’s planted missing-plan bug turns ForEach(Plan.all) into ForEach(Plan.all.prefix(2)). The paywall check on that build, on 28 September 2026:

✗ failed  The paywall shows three plans with Annual at $39.99 a year  (18.2 s)
  ✓ text "Choose your plan"
  ✗ count id=/^plan_/ == 3: found 2: plan_weekly, plan_monthly
  ✗ value id=plan_annual == "$39.99 / year": not found; closest: "plan_monthly"
  ✓ visible label=Restore Purchases role=button
  ✓ no_text "Loading"

When text or an element isn’t found, the line lists up to 5 near misses from the screen. That is what an agent needs to fix the right code, and what you need to trust the verdict without rerunning it.

How long it takes

Measured on 28 September 2026 on an M5 Max with macOS 26.5.1, Xcode 26.4 and the iOS 26.4 simulator, running Daybreak’s paywall check with five assertions:

A warm run spends about 1.5–2.5 s installing, 1.3–1.5 s launching and 1.4–1.7 s asserting. Simulators stay booted between runs, so the next run starts warm.

Make your app checkable

• Give the controls your checks depend on an identifier: .accessibilityIdentifier("plan_annual") in SwiftUI. An id selector then survives copy changes and translations.
• Mark selection as a trait: .accessibilityAddTraits(.isSelected). The selected selector reads it.
• Write each expectation so it only holds after the steps.

Write your first check

$ curl -fsSL https://mobster.dev/install.sh | sh
$ mobster sim doctor --fix
$ xcodebuild -scheme <Scheme> -destination 'generic/platform=iOS Simulator' \
  -derivedDataPath .mobster/build CODE_SIGNING_ALLOWED=NO build
$ mobster verify --app .mobster/build/Build/Products/Debug-iphonesimulator/<App>.app \
  --text "Welcome" --save first-check

Maestro vs XCUITest vs Mobster says when to use which, and Mobster vs Maestro and Mobster vs XCUITest compare them row by row.