Vibe Coding Tools

How to test an AI-built app before launch: a checklist (2026)

A pre-launch test plan for apps built with AI: the manual checks anyone can run, how to have the agent write automated tests that are worth having, and the payment, email and data tests people skip.

Floot ·

Key Takeaways

  • The preview only proves the happy path. Most launch bugs live in the paths you didn’t click: other users, signed-out visitors, bad input, small screens and slow networks.
  • Start with a written list of critical journeys. Sign up, sign in, the core action, payment, sign out. Test each one end to end on a fresh account before anything else.
  • Let the agent write tests, but specify them. Tell it the behaviours to cover, including the failures. Generic “add tests” requests produce tests that only confirm what the code already does.
  • Browser tests catch what unit tests miss. Tools like Playwright click through the real app like a user. Its codegen command records your clicks as a test you can keep.
  • Payments and email need their own tests. Use test mode, trigger webhook events deliberately, and check emails arrive outside the spam folder before real customers depend on them.

Quick-Start Prioritization Framework

TestWho runs itTimeCatches
Critical journeys on a fresh accountYou30 minBroken sign-up, core flow failures
Two-account permissions testYou15 minUsers seeing each other’s data
Signed-out and direct-URL testYou10 minUnprotected pages and APIs
Bad input and edge casesYou20 minCrashes, corrupted data
Phone and slow networkYou15 minLayout breaks, double submits
Automated tests for core rulesThe agent1–2 promptsRegressions after future changes
Payments, webhooks, emailYou + agent1 hourLost orders, missing emails

An AI-built app that works in the preview has passed exactly one test: the person who asked for it, signed in as themselves, on one device, following the happy path. Real users will do none of that. They’ll sign in on a second device, open links they shouldn’t, paste strange text, pay twice and close the tab mid-checkout. This guide is a pre-launch test plan for people who build with agents. It starts with manual checks anyone can run in an afternoon, then shows how to get the agent to write automated tests that actually protect you, and ends with the specialised checks for payments, email and data that most launches skip.

Why AI-built apps need deliberate testing

Agents are good at making the thing you look at work. They test, too: modern coding agents run commands and check output as they go. But they test what they think you asked for, and they can’t know the ways your users will misuse the app unless you tell them.

There’s a second reason. Each new prompt can change code that earlier features depend on. Without tests that run every time, you discover these regressions the way users do, by finding something broken that used to work.

Step 1: write down your critical journeys

Before testing anything, list the five to ten things the app must always do. For a booking app:

  • A new visitor can sign up and confirm their email.
  • A member can sign in on a second device and see their bookings.
  • A member can book a class with space and is refused for a full class.
  • A member can cancel their own booking.
  • An admin can see all bookings for a day.
  • A member can pay for a pass, and the pass appears after payment.
  • Signing out ends access, including on the back button.

Step 2: the manual checks anyone can run

These need no tools beyond a browser and a phone. Do them on the published version or a staging copy, not only the editor preview.

Fresh accounts, end to end

Use a new email address (many providers let you add +test1 before the @). Walk every critical journey from sign-up. Your own long-lived admin account hides problems like missing default settings or first-run errors.

The two-account test

Create users A and B. As A, create data. As B, open every page that shows data and paste A’s record URLs into the address bar. B must see none of A’s data. OWASP ranks Broken Access Control first in its 2025 Top 10, and this is the fastest way to find it.

Signed out

In a private window, visit admin, dashboard, settings and any API addresses you know directly. Each should redirect to sign-in or return an error, never data.

Bad and odd input

Empty required fields, a 5,000-character name, negative quantities, dates in the past, emoji, an apostrophe in a surname (O’Brien), a duplicate email at sign-up. The app should refuse clearly and save nothing broken.

Phone, slow network, double clicks

Open the app on a real phone. In a desktop browser’s developer tools, throttle the network to a slow profile and click submit twice quickly. Duplicate orders and bookings often come from exactly this.

Step 3: have the agent write automated tests

Manual testing finds today’s bugs. Automated tests stop them coming back after the next prompt.

Specify behaviours, not “add tests”

A vague request gets you tests that check whatever the code currently does, bugs included. Name the behaviours, including the failures: “Write tests for bookings: a member can book a class with space; booking a full class returns an error and creates nothing; member B can’t read or cancel member A’s booking; a signed-out request is refused. Run them and show me the output.”

Make sure the tests can fail

Ask: “Temporarily break the ownership check and run the tests. Confirm the permission test fails, then restore the check.” A test that never fails protects nothing. This takes one prompt and tells you whether the suite is real.

Run them on every change

Put the test command in your CLAUDE.md or AGENTS.md (“run npm test after every change and fix failures before finishing”). Anthropic’s Claude Code guidance recommends giving the agent a check like this so it can close the loop itself, and asking for the command and output as evidence.

Step 4: browser tests for real user flows

Unit tests check functions. Browser tests open the actual app and click through it, which is closer to what users experience.

Playwright in two commands

Playwright is a widely used open-source browser testing tool. Its docs give npm init playwright@latest to set it up and npx playwright test to run tests, with npx playwright show-report for an HTML report.

Record a test by clicking

Playwright’s codegen opens a browser and writes test code as you click: npx playwright codegen followed by your app’s URL. Record a critical journey once, then ask the agent to clean it up, add assertions (“the booking appears in My bookings”) and add it to the suite.

Let the agent write them directly

Or describe the journey: “Write a Playwright test that signs up a new user, books the 6pm class, and checks it appears in My bookings. Write a second test that signs in as another user and checks the booking isn’t visible.”

Step 5: payments, email and background jobs

These fail in ways no preview shows, because they depend on other services and on timing.

Payments in test mode

Run every payment flow in your provider’s test mode first. For Stripe, test successful payment, declined card, cancelled checkout and a refund. Then check your own database: did the order or subscription record update correctly in each case?

Webhooks, deliberately

Stripe’s docs say events aren’t guaranteed to arrive in order, that the same event may be delivered more than once, and that you should verify each webhook’s signature. The Stripe CLI can forward events to a local server (stripe listen --forward-to localhost:4242/webhook) and fire test events (stripe trigger payment_intent.succeeded). Ask the agent to test that a duplicate event doesn’t create a duplicate order and that an unsigned request is rejected.

Email that actually arrives

Send sign-up, password reset and receipt emails to Gmail and Outlook addresses. Check the spam folder, the links (do they point at the live domain, not a preview?) and how they look on a phone.

Scheduled jobs

If the app sends reminders or runs nightly tasks, trigger each job manually once and check the result. Then confirm it ran on schedule the next day by reading the logs.

Step 6: launch-day checks

The last pass, on the real domain, right before you share the link.

  • Every critical journey once more on the published app, with a fresh account.
  • HTTPS works on the bare domain and the www version.
  • Error pages exist and don’t show stack traces or internal details.
  • You can read the server logs and know where errors will show up.
  • You know how to restore the database and roll back the last deploy.
  • Test data and test accounts are removed or clearly separated.

Common mistakes

These are the testing gaps that most often surface as a user’s first support email.

  • Testing only as the admin or owner account.
  • Testing only the editor preview, not the published app on its real domain.
  • Accepting tests the agent wrote without checking they can fail.
  • Skipping the declined-card and duplicate-webhook cases.
  • Never trying the app on a phone before launch.
  • Leaving test payments, fake users or seed data in production.
  • Running tests once before launch and never again.

If you’d rather not do this yourself

If you’d rather not set up separate staging and hosting to test against, Floot shows every change in a live preview first and only updates the published app when you publish again, so you can run these checks before users see a change.

Frequently Asked Questions

How do I test an app I built with AI?

List your critical journeys, run them on fresh accounts, do a two-account permissions test, test signed out and with bad input, then have the agent write automated tests for the core rules and confirm those tests can fail.

Can the AI write the tests for me?

Yes, and it should. Specify the behaviours to test, including failure cases, and ask it to prove a test fails when the code is deliberately broken.

What is Playwright and do I need it?

Playwright is an open-source tool that runs a real browser to click through your app. It’s worth adding once you have a few critical journeys you don’t want to retest by hand after every change.

How do I test payments without spending money?

Use your payment provider’s test mode and test card numbers. For Stripe, the CLI can forward and trigger webhook events so you can test success, failure and duplicate events.

What’s the most commonly missed test?

Checking that one user can’t see another user’s data, including by pasting a record’s URL into a different account’s browser.

Should I test on the preview or the live app?

Both. Build and iterate on the preview, then run the critical journeys again on the published app and real domain before launch.

Sources