Let an agent verify your iOS app change before you merge the PR
A pull request changes the paywall and the settings screen. A reviewer reads the Swift and approves it, and nobody opens the app on a phone. Use mobster test pr to read the diff, find the saved checks that cover the changed screens, build the branch and run the checks on simulators and on a real iPhone plugged into your Mac. Use mobster ci init to write a GitHub Actions or GitLab CI workflow for that Mac as a self-hosted runner. In it, mobster ci run runs mobster test pr with video, and mobster ci comment keeps one comment on the pull request with the verdict and the command that reruns it.
From a diff to the checks it needs
mobster test pr runs the checks a branch needs, not the whole suite. The steps, from the testing docs:
- It reads
git difffrom where the branch leftorigin/main, uncommitted work included. - Each changed Swift view becomes a screen: its name, and the accessibility identifiers, strings and deep links on the changed lines.
- It picks every saved check that names a changed id or string, quotes a changed string in a step, carries the screen as a tag or file name, or was itself changed. When it picks none and an app file changed, it runs every check; a change to docs or tests alone runs the checks tagged
smoke. - It builds the app with
xcodebuild, for the simulator or, with--device, for the iPhone. - For each changed screen no check covers, it writes a draft check from the screen’s new ids and strings, at most 3. A draft runs beside the checks and never fails the pull request.
Steps 1 to 3 and 5 are plain code over a diff. No model reads your code. Here they are on a throwaway repository, run through the same functions mobster test pr calls. The branch renames a paywall button’s id and its title, and adds a toggle to the settings screen. Two saved checks exist: one for the paywall, one for a screen the branch doesn’t touch.
Changed screens: PaywallView, SettingsViewThe paywall shows three plans expects id upgrade_button expects “Choose a plan” names PaywallViewuncovered: SettingsViewPaywallView ids: upgrade_cta, upgrade_button texts: Choose your plan, Upgrade, Choose a planSettingsView ids: reminder_toggle texts: Settings, Daily reminderThe paywall check is picked for three reasons, and two of them are about the removed lines. The diff took upgrade_button and “Choose a plan” out, and a check that expects them is exactly the check that should run. The other check is left out: nothing it names changed. SettingsView has no check, so Mobster drafts one at .mobster/.
A check is never picked on a common word alone, so a PR that touches “Settings” in a string doesn’t wake every check that mentions it.
What the run prints
On a Mac with Xcode and a simulator runtime, from the branch:
mobster test prThe docs show the output for the same two changed screens in a sample app:
Changed screens: PaywallView, SettingsView✗ failed The paywall shows three plans on iPhone 17 Pro, iOS 26.4 text “Choose your plan”: not on screen; closest: "Choose a plan"~ draft .mobster/The failure says what the check expected and what the screen had. Mobster checks saved assertions (how a verdict is decided). The suite folder also holds pr.json (the changed screens, each check picked and why, the drafts), the frames with the expectations outlined, and a captioned GIF of each failed check. The exit codes are mobster test’s, counted without the drafts. A build that fails exits 3 and the comment quotes its first errors.
On a real iPhone, add the phone. mobster test pr --device "Test iPhone" builds for the phone and installs that build. With --testflight com.example.yourapp it skips the build and tests the TestFlight build already on the phone.
A Mac of yours is the runner
This setup runs the tests from a Mac, with the iPhone plugged into it. So the runner is a Mac you own, and a Mac mini on a shelf is the usual choice. Mobster hosts no runners or phones, and charges nothing per run or per device. Your code, builds and run artifacts are stored on your Mac and in your CI host’s artifacts. During model-driven steps, screen content also goes to your AI provider.
Setup is a standard user for the runner, automatic login, a Mac that never sleeps, Xcode, and the runner service as that user’s LaunchAgent. The docs’ page on a Mac as your runner has each step, Running checks in CI has the rest, and Run iOS UI checks in CI without a device farm walks through the workflow file and the reports. Then, in your project:
mobster ci initWrote .github/The workflow it writes runs on a runner with the mobster label, so your other jobs stay on hosted runners. The core of it is two commands: mobster ci run fetches the base branch and runs mobster test pr with --video, and mobster ci comment creates the comment or updates the one it made on an earlier push. The comment never fails the job. GitLab gets the same job in .gitlab-ci.yml.
mobster ci doctor tells you what the Mac still needs before the first job. This is the docs’ example of a runner with two things to fix:
✓ Apple silicon this Mac is arm64✓ Xcode Xcode 26.4✓ iOS Simulator runtime iOS 26.4✓ Mobster for Mac in Applications✓ Mobster plan active on this Mac✗ System sleep after 10 minutes Run `sudo pmset -a sleep 0 disksleep 0`, so the Mac never sleeps through a job.! Restart after a power cut off Run `sudo pmset -a autorestart 1`.✓ Automatic login as ci✓ Runner service GitHub Actions runner✓ Free disk 212 GB✓ Phones this runner allows none: checks run on simulators onlyA row marked ✗ fails and names the fix. A row marked ! works and is worth fixing.
The comment
Each push updates one comment. A row per check, with what it expected, what it found and the frame. A run on one device names it once, after the commit:
### Mobster checks: 11 passed, 1 failedChanged screens: `PaywallView`, `SettingsView`, at `a6d8d39` on iPhone 17 Pro, iOS 26.4| Check | Expected | Found | Frame || --- | --- | --- | --- || **Failed** The paywall shows three plans | text “Choose your plan” | not on screen; closest: "Choose a plan" | paywall.jpg || **Passed** Marking a task done lowers the count | 3 expectations | as expected | tasks.jpg |Each check that didn’t pass gets a folded repro: the device, the deep link it opened, the steps it took, what it expected and found, its video and GIF, and the command that runs it again on the same device. Up to 5 checks get one. Typed text, email addresses and phone numbers in a step show as •••. GitHub doesn’t show images from a workflow’s artifacts inside a comment, so frames, GIFs and videos are links to the uploaded report.
With Mobster Checks, the Mobster GitHub App, installed on the repository, the verdict also shows as a check named Mobster: success when every check passed, failure when one failed or needed review, neutral when no saved check covers the pull request. Add Mobster as a required check in the branch’s protection rules to block a merge on a failure.
Your iPhone, and who may reach it
A pull request can edit the workflow file. So a phone named only in the workflow doesn’t reach it. The runner’s owner lists the phones it may test, and any TestFlight apps, in the runner’s own .env. These settings are separate from the workflow file; pull-request code still runs as the runner user:
# ~/ - name: Test what this PR changed run: | "$MOBSTER" ci run --device "Test iPhone" --testflight com.example.yourapp- A
--deviceor--testflightthe runner doesn’t list stops the job before a check runs. - On a phone, a check drives only builds you installed: the device build from the job, a build Xcode installed, or a TestFlight build you named. An App Store app can’t run.
- Smart’s steps stay in your app, and never send, buy, post or delete on the phone.
- A replayed step that sends, posts, deletes or buys asks the same approver a live step asks, and on an iPhone every commit is refused. The safety docs list what counts.
- With several phones, list each in
MOBSTER_RUNNER_DEVICESand add--spreadso each check runs once on whichever phone is free. - Use a test phone signed in to a test account. Keep it unlocked, awake and charging.
One rule you can’t configure around: a self-hosted runner runs the code of every pull request it picks up, as the runner’s user, on the Mac the phone is plugged into. Don’t connect one to a public repository that takes pull requests from forks. GitHub’s own hardening guide for self-hosted runners explains why.
When the agent is your coding agent
Your coding agent can start the same run. Over MCP, run_tests with pr: true reads the diff, picks the checks, drafts the missing ones, builds, runs, and answers with the verdict, each check’s frame and GIF, the failing frame as an image and the comment’s path. A long build answers with a suite_id that the agent waits on.
The phone is a separate decision, and the server makes it, not the agent. pr: true runs on a real iPhone only when the server was started with --allow-device naming that phone, and tests a TestFlight build only when --allow-testflight names the app too. Without the flags the device argument is refused, so an agent can’t widen either. The simulator needs no flag. AI end-to-end tests for iOS covers the loop and the assertions, and the open-source Mobster engine covers the code that stops an agent at a send button. The MCP server and the mobster command are free and open source too: install them with curl -fsSL https://mobster.dev/install.sh | sh, or use the ones inside Mobster for Mac.
Set it up
Install Mobster for Mac on the runner, run mobster ci doctor, then mobster ci init in your project and open a pull request. The first run of a check with steps needs a model key. Launch-only checks, the ones with no steps, need none. The CI docs have every flag, including --shard to split a suite across Macs.
Questions
Can I test an iOS pull request on a real iPhone in CI?
Yes, on a Mac of yours. Plug the iPhone into a self-hosted runner, list its name in MOBSTER_RUNNER_DEVICES in the runner's own environment, and run mobster ci run --device with that name. Mobster hosts no runners and no phones.
Does Mobster run every check on every pull request?
No. mobster test pr reads the diff, picks the saved checks that name a changed accessibility id or string, and runs those. When it picks none and an app file changed, it runs every check; a change to docs or tests alone runs the checks tagged smoke.
Can a pull request point the runner at a different phone?
Mobster checks the device against the runner's own .env file. A device named only in the workflow file stops the job before a check runs. Pull-request code still runs as the runner user.
Can my coding agent run this on my iPhone?
Over MCP, run_tests with pr true runs on a real iPhone only when the server was started with --allow-device naming that phone. Without the flag the device is refused.
