AI app builders have changed who can make software. With Lovable, Bolt, Replit or Cursor, a founder with no coding background can have a working app in a weekend. I use AI to build too: I built Craftura, a full-stack furniture e-commerce site, with heavy AI help, and wrote up everything it got wrong.
But "it works when I click through it" and "it's safe to put real customers on" are different tests. AI-built apps tend to pass the first and quietly fail the second, because the problems don't show on screen. Here are four checks you can run yourself, what each one protects you from, and what to ask a developer if one fails.
Why this matters now
Security researchers have spent the last year scanning AI-built apps, and the same problems keep turning up. In one case, recorded as CVE-2025-48757 and rated critical, a researcher scanned 1,645 apps from Lovable's public showcase and found 170 whose database tables could be read without proper access rules. Lovable disputes that the flaw is theirs, arguing that each app's owner is responsible for protecting its data. Either way, the owner is the one who carries the risk.
It isn't one tool's problem. In 2025, Veracode tested more than 100 AI models on 80 coding tasks built around known security weaknesses. The code worked, but in 45% of cases the model chose the insecure way to write it, and bigger models didn't do noticeably better.
AI writes code that works. Whether it's safe is a separate question, and nobody asks it unless you do.— Kathan N. Patel
Check 1: Can one user see another user's data?
This is the big one. Most apps keep every customer's data in the same database, and the only thing stopping customer A from reading customer B's orders is an access rule. If that rule is missing, or only enforced in the browser, anyone who knows where to look can read everything.
On Craftura, the AI-generated code that protected the admin panel looked completely reasonable when I read it, and it crashed in production. I caught it because I knew what to look for. If it had half-worked instead, the admin panel would have looked protected without being protected.
How to check it yourself: create two test accounts. Log in as the first, open something that belongs to it (an order, a profile, an invoice) and copy the address from the browser bar. Log in as the second account and paste that address. If you can see the first account's data, stop and fix it before launch. Then try the admin pages while logged out.
What to ask a developer: "Where is access checked: in the browser, or on the server and in the database?" The right answer is the server or the database, for every table. If your app uses Supabase, ask whether Row Level Security is switched on for every table, with rules that match who should see what.
Check 2: Are any secrets sitting in the browser?
Apps talk to other services with keys: payment providers, AI APIs, email services, the database itself. Some keys are meant to be public. Others must never leave the server, because anyone who copies them can spend your money or read your data. AI tools sometimes put the second kind where the first kind belongs, because that's the quickest way to make something work.
How to check it yourself: open your app in Chrome, press F12, go to the Sources tab, and search all files (Ctrl+Shift+F) for words like secret, sk_, sk-, service_role and api_key. Then search your code repository for the same words, including old commits. Anything that looks like a long random string next to those words needs a developer's eye.
What to ask a developer: "Which keys are in the browser code, and what can each one do?" A Supabase publishable key (the older name is "anon key") in the browser is normal, but only if Row Level Security is on, which is check 1 again. A Supabase secret or service-role key is never normal there: Supabase's own documentation warns that it "bypasses every Row Level Security policy you have." The same goes for a Stripe secret key or an AI provider's key. Any key that was ever in a public repository should be replaced, not just deleted.
Check 3: What happens when something fails halfway?
AI-generated code is usually written for the happy path: the user clicks once, the network holds, every service answers. Real life is messier. People double-click. Phones lose signal mid-checkout. Payment providers resend notifications: Stripe's own documentation says webhook endpoints "might occasionally receive the same event more than once."
When the code doesn't expect that, you get double orders, duplicate customers, emails sent twice, or a status that never updates. On Craftura, the AI-written order tracking page showed a cached status, so customers refreshing it could see a status that was hours old.
How to check it yourself: in a test environment, double-click every button that creates something (sign-up, order, payment) and count what gets created. Refresh during checkout. Switch your phone to airplane mode halfway through a form, then back. Then look in your admin panel or database for duplicates.
What to ask a developer: "What happens if the same payment notification arrives twice?" A good answer mentions recording each event's ID and ignoring repeats, or a unique constraint in the database. "We check whether it exists first" is weaker, because two requests arriving at the same moment can both pass that check. I explain that trap in more detail in this post on integrations and duplicate records.
Check 4: Can anyone change it safely next month?
An app isn't finished at launch. You'll want changes, and every change to code nobody understands risks breaking something else. AI tools make this worse in one specific way: they generate a lot of code very quickly, so an app can outgrow anyone's understanding of it within weeks.
How to check it yourself: answer four yes-or-no questions. Is the code in Git (for example on GitHub), under an account you own, with its history? Is there a backup of the database, and has anyone actually restored one? Can a fix be deployed, and undone, without taking the app down? Are there any automated tests at all, even a few for sign-up, login and payment?
What to ask a developer: a one-page handover covering where the code lives, how it's deployed, which services and keys it uses, and how to restore a backup. If nobody can write that page, that's the finding.
When to bring in a developer
You don't need one to build a prototype, test an idea or show investors. You do need one before any of these: taking payments, storing personal data (health, legal or financial information above all), opening sign-ups to the public, or connecting the app to systems your business depends on. Call one too when the AI keeps fixing one bug by creating another. That loop usually means the structure underneath has stopped making sense.
A review doesn't have to mean a rebuild. The useful work is usually targeted: lock down the access rules, move the secrets, make risky operations safe to repeat, and write the handover page. The goal is to keep what works and fix what's dangerous.
If you've built something with AI and want a second pair of eyes before launch, book a free 30-minute call. Bring the app and your answers to the four checks.
Sources
Veracode, 2025 GenAI Code Security Report (July 30, 2025)
NIST NVD, CVE-2025-48757
Matt Palmer, Statement on CVE-2025-48757 (the researcher's own account of the scan)
Supabase, Understanding API keys
Stripe, Handle duplicate events
