How to set up automated regression testing for an iOS app with a small team

The options side by side

You can mix them. In the next release, you run your XCTest targets on a simulator with mobster test --xctest and put their results in the same report as your checks.

Step 1: Save a check for each flow

A Mobster check is a YAML file: the app, the steps in plain English, and what must be on screen at the end. Mobster checks the saved assertions.

Build your app for the simulator, then run the flow once with --save:

$ xcodebuild -scheme YourApp \  -destination 'generic/platform=iOS Simulator' \  -derivedDataPath .mobster/build \  CODE_SIGNING_ALLOWED=NO build$ mobster verify "Tap Settings, then tap Upgrade." \  --name "The paywall shows the plans" \  --app .mobster/build/Build/Products/Debug-iphonesimulator/YourApp.app \  --text "Choose your plan" \  --env-file ~/.config/mobster/agent.env \  --save paywall

The agent built into Mobster performs the steps on your Claude or OpenAI API key. --env-file names a file with one line, ANTHROPIC_API_KEY=… or OPENAI_API_KEY=…. --save paywall writes the check to your project:

# .mobster/checks/paywall.yamlversion: 1name: The paywall shows the plansapp:  path: .mobster/build/Build/Products/Debug-iphonesimulator/YourApp.appsteps:- Tap Settings, then tap Upgrade.expect:- text: Choose your plan

• Commit the file with your code.
• A launch-only check, with no steps, never calls a model.
• Add tags: [smoke] to your most important checks. mobster test --tag smoke runs only those, and mobster test pr runs them when a branch changes no screen.
• If you have Maestro flows, turn them into checks with mobster test import maestro .maestro.

Step 2: Run every check on simulators and your iPhone

$ mobster test --sim "iPhone 17 Pro" --sim "iPhone SE (3rd generation)" --parallel 2$ mobster test --device "Test iPhone"

• mobster test runs every check in .mobster/checks, each on its own simulator. The default is an iPhone 17 Pro.
• Every check runs on every --sim. --parallel runs up to 8 simulators at once, fewer on a Mac with less memory or fewer cores.
• --device adds an iPhone from mobster devices, and --device all adds every iPhone that is plugged in, unlocked and set up. Each phone runs one check at a time. Keep the phone unlocked.
• On a phone, a check drives only a build of your own app. Mobster never clears the app’s data there, and it refuses every send, buy, post and delete.
• Mobster runs a failed check once more, and marks it flaky if it passes on that try. A check listed in .mobster/quarantine.yaml keeps running, and its ordinary failures don’t fail the run.
• Each run writes junit.xml and an HTML report with the frame behind each verdict. The exit code is the verdict: 0 passed, 1 failed, 2 needs review, 3 couldn’t run.

Step 3: Replay with no model call

The first time a check with steps passes, Mobster keeps the exact steps its agent took. From then on, mobster test replays them through WebDriverAgent with no model call, for $0. Mobster checks the saved assertions after a replay, as it does after a fresh run.

The measurement covers a two-step check in a test app on an iPhone 15 Pro with iOS 26.0.1 over USB. Median times are 17.7 s fresh and 10.3 s replayed. Model cost is about $0.017 per fresh run and $0 per replay.

• A changed screen. When a control moves or its name changes, the agent redoes that step on your API key. The replay then continues. mobster test heal shows each heal with the frames before and after, for you to accept or reject. A heal never changes an expectation. A check still fails if the app breaks that expectation.
• A new build. In Mobster 0.4.0 on a simulator, a new app build, iOS major version or Mac records the path again, on your key. In the next release a new build replays, and a new device model records again. A step whose control changed heals as above.

Step 4: Write a pull-request workflow with mobster ci

mobster ci runs on a Mac you own, registered as a self-hosted runner. The runner is an Apple silicon Mac on macOS 13 or later, such as a Mac mini. It needs Xcode with an iOS Simulator runtime, and Mobster for Mac signed in. Run iOS UI checks in CI without a device farm sets one up.

From your project’s folder:

$ mobster ci init

It writes .github/workflows/mobster.yml, or a mobster job in .gitlab-ci.yml on GitLab, for a runner with the mobster label. Add --device "Test iPhone" to mobster ci init to test on an iPhone plugged into the runner too. Add ANTHROPIC_API_KEY as a repository secret for checks with steps. On each pull request, mobster ci run runs mobster test pr --video:

1. It reads the branch’s diff and picks the saved checks that cover the changed screens.
2. It builds the app with xcodebuild and runs those checks.
3. It drafts a check for each changed screen that no check covers, at most 3. A draft never fails the pull request.
4. It fails the job when a check fails or couldn’t run.

mobster ci comment keeps one comment on the pull request up to date. You get the verdict and, for each failure, the steps to reproduce it, links to the frame, a GIF and the video, and the command that runs it again.

A phone the runner doesn’t list stops the job before a check runs. List it in the runner’s own .env file, which no pull request can change, then restart the runner:

# ~/actions-runner/.env$ MOBSTER_RUNNER_DEVICES=Test iPhone

A self-hosted runner runs the code in each pull request. Don’t use one on a public repository that takes pull requests from forks, and use a test phone signed in to a test account.

To run every check each night on every iPhone plugged into that Mac:

$ mobster test nightly on --at 02:00 --device all

Each morning, .mobster/nightly/<date>/morning.md starts with the verdict. It lists each check that failed or was flaky, with the command that runs it again.

What it costs

Runs on a real iPhone and mobster ci run need Mobster for Mac. It is free on any number of iPhones. Fresh and healed steps use your Claude or OpenAI API key, billed by your provider. Replaying recorded steps makes no model call.