How do I know whether an AI-built app is ready to launch?

Require evidence for each promised workflow, real data persistence, authorization boundaries, failure behavior, production builds, monitoring, backup, and rollback. A passing demo or an agent saying done is not sufficient evidence.

Is a code scan enough to approve an AI-built app?

No. Static analysis can find useful signals, but launch approval also requires functional tests, negative tests, production-like execution, dependency review, and human judgment about the application's actual requirements.

What should block an AI-built app from launching?

Block launch when a critical workflow lacks evidence, authorization can be bypassed, secrets are exposed, real writes or payments are unverified, errors falsely report success, production cannot be rolled back, or the team cannot explain what remains untested.

// proof-first pre-launch guide

How to audit an AI-built app before launch

A polished demo proves that one path worked once. A launch decision needs evidence that the promised workflows are real, data is protected, failures stay honest, and the team can recover when production behaves differently.

By H.J. Westerfield · Updated July 20, 2026 · 11 minute read

The short version

Write down what the app claims to do. For every critical claim, record a reproducible test and its evidence. Search for unfinished or fake-success code, verify real writes and reads, test unauthenticated and cross-user access, force dependency and network failures, run the production build in a production-like environment, then prove monitoring, backup, and rollback. Anything unverified stays visible and can block launch.

A convincing demo is not a release artifact

AI coding tools are very good at completing the path described in the prompt. That can hide a different class of incomplete work: a route that returns sample data, a button that reports success without checking the write, an empty state that was never rendered, or an authorization decision performed only in the browser.

The right response is not to distrust every generated line. It is to make the acceptance standard independent of who wrote it. GitHub's responsible-use guidance says generated code can contain vulnerabilities or other issues and should be reviewed. The OWASP Secure Coding with AI guidance similarly warns against letting generated code or generated review comments replace human review.

NIST's minimum verification guidance recommends combining threat modeling, automated tests, static analysis, secret detection, black-box and structural tests, fuzzing where appropriate, and review of included components. No one green badge covers the whole decision.

The seven release gates

Gate	What must be true	Strong evidence
Completeness	Every promised critical workflow exists beyond the happy-path UI.	Acceptance tests, route inventory, and no unexplained mocks, stubs, placeholders, or fake-success branches.
Data reality	Writes persist, reads return the right tenant's data, and migrations match production.	A fresh-account test, database evidence, reload verification, migration dry run, and cross-user isolation test.
Failure honesty	Errors do not masquerade as success or leave partial state behind.	Forced network, timeout, expired-token, duplicate-submit, and dependency-failure tests with visible outcomes.
Authorization	The server enforces who can perform each sensitive action.	Unauthenticated, wrong-role, and cross-account negative tests at the API or server boundary.
Supply chain	Secrets, dependencies, build scripts, and runtime configuration are understood.	Secret scan, dependency inventory, lockfile, vulnerability review, and production variable checklist.
Production fit	The deployable artifact works under production configuration and realistic data.	Clean production build, staging smoke test, performance budget, monitoring event, and accessibility check.
Recovery	The team can detect, contain, reverse, and learn from a bad release.	Named owner, backup restore evidence, rollback procedure, alert destination, and recorded residual risks.

The proof-first audit workflow

Freeze the claims before reading the code

List the user-facing promises, critical business rules, data boundaries, integrations, and irreversible actions. Turn vague claims such as “payments work” into observable outcomes: the correct amount is charged once, the order is recorded, the user receives the right status, a failed webhook can retry safely, and a refund follows the documented path.

This prevents the audit from shrinking to whatever the current implementation happens to support.

Inventory what actually runs

Map entry points, routes, background jobs, database migrations, third-party calls, environment variables, build scripts, and deployment configuration. Compare that inventory with the claims. An attractive screen with no durable write path is not a completed feature.

rg -n -i "todo|fixme|hack|not implemented|coming soon|placeholder|mock|fixture|sample data" .
rg -n -i "return true|success: true|status: ['\"]ok|catch\s*\([^)]*\)\s*\{\s*\}" .

These searches create a review queue. Test fixtures and explanatory comments may be legitimate; unexplained production paths are not.

Prove the real data path

Create a new account with no seeded state. Complete the workflow, reload, sign out and back in, then inspect the stored record through an independent surface. Repeat with a second account and attempt to read or change the first account's object. Check deletes, edits, duplicate submissions, and concurrent updates—not only creation.

For payments, email, file uploads, queues, and webhooks, verify the provider-side event and your own resulting state. A toast message is not evidence that the remote action completed.

Force the paths the prompt probably omitted

Test empty input, maximum input, malformed input, an expired session, a denied permission, a slow response, a network loss, a provider 500, a retry, a double click, and a partial dependency outage. Confirm that the user sees an honest state and the system does not silently continue.

This aligns with NIST's risk-based approach: test the executable artifact, document the results, triage findings, and include the real production infrastructure in the test plan.

Verify authorization at the server boundary

Authentication proves identity; authorization decides whether that identity can perform this action. OWASP recommends server-side access-control checks and explicit handling of failed checks. Test every sensitive route without a session, with the wrong role, and with another user's object identifier.

Do not accept a hidden button, a client-side guard, or an unguessable URL as the control.

Review the supply chain and configuration

Run the project's tests, production build, linter, type checker, static analyzer, secret scanner, and dependency audit where applicable. Explain every install-time script and direct dependency. Confirm that production secrets never reach browser bundles or logs, and that staging and production use separate credentials and data.

NIST's current DevSecOps reference model combines peer review with software-composition analysis, static testing, linting, functional testing, and security testing rather than treating any one tool as the release gate.

Run the artifact you will ship

Build from a clean checkout and deploy the same artifact to a production-like environment. Exercise the critical workflows on desktop and mobile, with realistic data volumes and a new-user state. Capture browser and server errors, response codes, job failures, and performance regressions. Confirm analytics and alerts with a known test event.

Prove recovery, then make the decision

Restore a backup into a safe environment, execute the rollback path, and name the person who will respond to a failed release. Record every unverified item and residual risk. Launch only when the decision-maker can see what passed, what failed, what remains unknown, and why the remaining risk is acceptable.

Use an evidence ledger, not a completion percentage

A completion score hides the difference between seven cosmetic checks and one missing authorization boundary. Keep one row per claim or risk, and attach the artifact that another reviewer could inspect.

Minimum release-evidence fields

Claim: the user-visible or operational behavior being approved.
Risk: what happens if the claim is false.
Test: the exact action, input, environment, and expected result.
Evidence: test output, screenshot, database record, provider event, log, or deploy ID.
Status: pass, fail, blocked, or unverified. Never turn “not checked” into “probably fine.”
Owner and date: who reviewed the result and when it becomes stale.

| Claim | Risk | Test | Evidence | Status | Owner |
|---|---|---|---|---|---|
| User A cannot read User B's record | Data exposure | Call record API as User A with User B ID | test log + 403 response | pass | H.J. |
| Failed payment is not saved as paid | Revenue/support | Force provider decline | provider event + DB row | unverified | — |
| Release can be rolled back | Extended outage | Restore previous deploy | deploy ID + smoke test | pass | H.J. |

The ledger is intentionally blunt. It makes an honest “unverified” more valuable than a confident summary with no inspectable proof.

Conditions that should block launch

A critical user or revenue workflow has no reproducible evidence.
Production code still returns fixtures, placeholder content, hardcoded success, or an unimplemented branch.
A client-side control is the only barrier to another user's data or an administrative action.
An error is swallowed, converted to success, or leaves a partially completed payment, order, or migration.
A secret appears in source, browser output, build artifacts, logs, or a public repository.
The production build has not been run from a clean environment with production-like configuration.
No one can show a working backup restore or rollback path.
The release decision depends on the same agent that wrote the code declaring its own work complete.

This checklist is a practical release screen, not a security certification or a substitute for specialist review of high-risk systems.

Make the repeatable parts easier to rerun

The audit still needs human judgment. Focus automation on repeatable evidence collection and high-signal review queues.

AI Build Sanity CheckRead-only triage for mocks, placeholders, swallowed errors, unfinished branches, and endpoints that report success without proving it.See the flagship check → Test Quality AuditorFind weak assertions, skipped cases, over-mocking, and tests that exercise implementation without proving behavior.Audit the test evidence → PII & Data-Leak ScannerReview schemas, seed data, configuration, and logs for personal data that should not reach the wrong environment.Check the data boundary →

Common questions

How do I know whether an AI-built app is ready?

Require inspectable evidence for each critical claim: the workflow, its real data path, authorization boundary, failure behavior, production artifact, monitoring, and recovery path. A demo and a completion message are inputs, not approval.

Is a static scan enough?

No. Static checks are useful for unfinished patterns, secrets, dependencies, and common bugs. They cannot prove real integrations, tenant isolation, runtime failure behavior, production configuration, or recovery.

Do AI-generated features need different standards?

The acceptance standard can stay the same: risk-based review, testing, and evidence. AI-assisted work often increases the need to make hidden assumptions and unverified completion visible, but human-written code can fail the same gates.

What if I am not a developer?

Start with observable outcomes and a separate test account. Ask for evidence another person can inspect: a failed authorization request, a persisted record after reload, a provider event, a clean production build, and a demonstrated rollback. Bring in a qualified reviewer for sensitive data, payments, regulated use, or any behavior you cannot verify.