Key takeaways
- Agents test the paths they were told about; TestFlight testers find the rest.
- Internal testers get builds as soon as processing finishes; external testers need a beta app review first.
- The App Store Connect API exposes TestFlight screenshot feedback and crash submissions, so an agent can help with triage.
- Every report becomes one issue with steps to reproduce before an agent touches the code.
- Written exit criteria turn "ready for App Review" into a decision rather than a feeling.
Why agent-built apps need a real beta
A coding agent verifies its work with the checks it can run: the compiler, unit tests, perhaps UI tests on a simulator. All of them start from assumptions written into a prompt. Testers start from none. They install on an older iPhone, deny the notification prompt, lose signal in a lift and paste an emoji into the name field. Those paths produce the crashes and dead ends a reviewer finds – and a crash on the reviewer's device is the quickest route to a Guideline 2.1 rejection.
There is a second reason. Apps built in many agent sessions accumulate behaviour nobody planned: a settings screen that resets data, a retry loop that hammers the backend, copy that contradicts itself. People notice these within minutes; automated checks often never do.
Setting up groups and builds
| Internal testing | External testing | |
|---|---|---|
| Who | Members of your App Store Connect team, up to 100 | Anyone invited by email or public link, up to 10,000 |
| Review | None – available after processing | Beta app review before the first build reaches testers |
| Best for | Every build from the main branch | Release candidates and real target users |
Let CI upload every main-branch build to the internal group automatically – the fastlane playbook has a lane for it – and promote selected builds to external groups by hand. Builds expire after 90 days, so a long beta needs fresh uploads anyway. External testing also needs test information in App Store Connect: a description, a feedback email address and, for apps with login, a demo account the beta reviewer can use. Later builds of the same version may not need a full review again, but do not plan around that.
All of this assumes the build archives with Xcode 26 on a Mac or a macOS CI runner. Cloud agents that run on Linux can prepare the code; they cannot produce the build.
Talk to an AI Code Engineer about beta testing your app with TestFlight.
In a free call we look at what you built, tell you honestly what stands between it and an approved release – and whether you need us at all.
What to Test notes the agent drafts
Every build can carry What to Test notes. An agent drafts them well from the pull requests merged since the previous build: what changed, which screens to try, what is known to be broken. Edit before sending – testers follow notes literally, and vague notes produce vague feedback. A useful structure per build:
- two or three concrete tasks, such as "create a list, share it with a second account, then delete it",
- the fixes testers should re-check, named so the people who reported them recognise them,
- known issues, so nobody spends time reporting them again.
For the first external round, add a short test plan that follows the reviewer's path: install, sign up, log in, use the core feature, buy something if the app sells anything, delete the account.
The feedback loop: from screenshot to agent task
Testers send feedback from the TestFlight app with a screenshot and a comment, and crash reports arrive on their own. Both appear in App Store Connect, and the App Store Connect API exposes screenshot feedback and crash submissions, so a small script – or an agent with API access – can collect them on a schedule. A loop that holds up:
- Collect new feedback and crashes once a day.
- Group. The agent clusters reports that describe the same problem and attaches device model, iOS version and build number.
- Write issues. One issue per problem: steps to reproduce, expected and actual behaviour, the original screenshots, and a symbolicated stack trace for crashes.
- Prioritise. A person decides what blocks the release.
- Fix. The agent takes one issue per branch, adds a test that fails before the fix, and links the pull request to the issue.
- Close the loop. The next What to Test notes name the fixed issues so testers can confirm them.
Keep a person between grouping and fixing. Tester comments are text from outside your team; the agent should read them as bug descriptions, never as instructions to follow.
Getting through the beta app review
The beta review is lighter than App Review but checks the same basics: the app launches, the reviewer can sign in, and nothing is obviously broken or objectionable. Agent-built apps tend to trip over simple things here – a demo account that needs an SMS code, a staging backend that went to sleep, a developer menu still visible in the build. Run the code scan from our App Store review checklist before the first external build; it catches most of them.
Do not treat external TestFlight as a permanent way to distribute the app. It is a test channel with expiring builds, and real users belong on the App Store.
Exit criteria: when the beta is done
Decide in advance what "ready" means, store it in the repository and let the agent report against it before each release candidate:
- no open crash in the last two builds on any supported iOS version,
- every blocking tester issue closed and confirmed by the person who reported it,
- the full reviewer path completed by at least two people on two device types,
- App Privacy answers, privacy manifest and review notes matching the final build.
Then submit exactly the build the testers approved. Android teams run a similar loop – with a fixed tester count for new personal accounts – as described in Google Play closed testing for AI-built apps. If you want help running the beta and the submission, our AI Code Engineers handle TestFlight distribution and App Review in your own account; the iOS submission service describes how.
TestFlight setup checklist
- CI uploads every main-branch build to an internal testing group automatically.
- External groups contain people who resemble your real users, not only colleagues.
- Test information includes a description, a feedback address and a working demo account.
- Each build carries edited What to Test notes with concrete tasks and known issues.
- Feedback and crashes are collected daily and grouped into issues with steps to reproduce.
- dSYMs are available so crash reports can be symbolicated.
- Fixes are linked to issues and named in the next build's notes.
- Written exit criteria decide when a build goes to App Review.
Frequently asked questions
Do internal TestFlight testers need an App Store Connect account?
Yes. Internal testers are users on your App Store Connect team with an appropriate role. Everyone else joins as an external tester through an email invitation or a public link, after the build has passed the beta app review.
Can a coding agent upload builds to TestFlight?
It can run the upload lane in CI once a person has set up signing and an App Store Connect API key. Creating tester groups and deciding which build goes to external testers are better left to a person.
How long does the TestFlight beta review take?
Apple does not commit to a fixed time. Plan a buffer before the date you need external testers, and make sure the demo account and test information are complete so the review is not delayed by questions.
Can we use TestFlight instead of publishing on the App Store?
No. TestFlight builds expire after 90 days and the service is meant for testing. For distribution to real users, the app goes through App Review and onto the App Store.
Can you run our TestFlight beta?
Yes. Book a free call and an AI Code Engineer sets up builds, groups and the feedback loop with you, and can take the app through App Review afterwards.
Free, 15 minutes, no obligation
Get your app from agent output to approved release.
Our AI Code Engineers take apps built with Claude Code, Codex, Cursor, Lovable, Base44 and co. through signing, CI/CD, store listings and review – in your own developer accounts.
AI tools, store policies and requirements change quickly – always check the current documentation of the tools and stores you use. This article is independent advice and not affiliated with or endorsed by Apple, Google, Anthropic, OpenAI or any other company or product mentioned; all trademarks belong to their owners.