AI Build Sanity Check

What it looks for

• Configured build-marker terms such as TODO, FIXME, “not implemented,” and NotImplementedError.
• Return lines containing configured fake, mock, sample, dummy, or placeholder terms.
• Python except blocks followed by pass and same-line empty JavaScript catch blocks.
• Configured placeholder strings in supported source, config, and .env files.
• Return lines containing success, ok, or status near a supported route marker.
• Route-adjacent empty or throwing handler markers as advisory review prompts.

This is line- and term-based static analysis, not proof that a behavior is unfinished. The bundled Python 3 scanner is standard-library only and read-only; its findings create a review queue and do not authorize an automatic rewrite.

See four findings from one supported route

This example uses the exact route, return-term, build-marker, and same-line catch forms recognized by the packaged scanner.

export async function POST(request) {
  try {
    // TODO: replace demo response with database write
    return Response.json({ ok: true, orderId: "demo-123" });
  } catch (error) {}
}

Supported application files

The scanner reads .py, .js, .ts, .jsx, .tsx, .html, .json, .env, and files named .env inside the selected paths. Because .env is included, scope the run away from it if you do not want that file read.

Where it helps

• Auditing an AI-generated MVP before a client demo or production push.
• Checking a pull request for supported marker, placeholder, error, and route-return patterns.
• Reviewing whether configured fake-data terms appear on return lines in the real data path.
• Giving a non-coder a plain-English list of suspicious shortcuts to take back to the builder.
• Creating a focused first pass before deeper functional, security, authorization, and recovery testing.

How to run

Get it on Agensi and use an agent that reads SKILL.md, can run Python 3 locally, and can read the selected paths. Run python scripts/scan_ai_build_sanity.py <repo-or-folder>, then review deterministic and advisory findings before deciding what to change.

Read-only by design

The scanner reads supported source and configuration files without modifying or deleting them. It reports file, line, rule, severity, and a generic message—not the matched line or value. It does not install packages, start the app, write files, or call the network.

It is a heuristic scanner, not formal verification. A clean report does not guarantee logical correctness, security, real integrations, tenant isolation, production behavior, or recovery.

Use it inside a release decision

The proof-first AI-built app audit shows how to turn findings into evidence for completeness, real data, failure paths, authorization, production behavior, and rollback.

Questions