It's slightly the wrong question
Human-written code was never secure. That's the whole reason SAST, code review and penetration testing exist, and they all existed long before anyone had heard of a language model. Models learned from that code, so they reproduce the same mistakes in it.
So in principle, no. AI-generated code is about as good as the architecture it was written into and the review it came out of, which was always true of the rest of it.
Our own numbers say something less comfortable.
24 apps, one clean
We built 24 applications across Lovable, Replit and Manus and scanned all of them. Twenty-three came back with vulnerabilities. 561 in total, so roughly 23 per app. The Replit builds averaged 26 or more.
We expected a spread. Some sloppy, some fine, more or less what you'd get from a team of mixed experience. That isn't what happened. Nearly everything failed, and it failed in the same way each time.
Pattern | CWE | What it looked like |
|---|---|---|
Broken object-level auth | CWE-639 | Auth check passes, but the query never filters by the logged-in user |
Hardcoded credentials | CWE-798 | API keys and database tokens sat in client-side code so the preview would work |
No rate limiting | CWE-307 | Login, password reset and OTP endpoints with no throttling |
Permissive CORS | CWE-942 | Wildcard origins set so the preview pane renders, shipped unchanged |
Auth bypass | CWE-287 | Tokens issued correctly, gap in the validation logic lets unauthenticated requests through |
Look at what those have in common. Not one of them is a mistake in the code that's there. They're all things that aren't there, a missing filter on a query, a missing throttle on a login route, a missing branch in a token check. The code runs fine. It only falls over when somebody attacks it.
The defaults, not the model
Platform scaffolding is built so the preview works first time. Wildcard CORS so the iframe renders. Keys client-side so the demo has data. No rate limit, because the tutorial it learned from didn't have one either.
Every one of those is a reasonable thing to do while you're building and wrong the moment it goes live. The model isn't making an error. It's copying tutorial code accurately, and tutorial code was never written with an attacker in mind.
Volume still matters. You can generate an app in an afternoon and nobody's review capacity grew to match, so most of this ships without anyone reading it. But volume on its own would have given us a distribution. Defaults gave us 23 out of 24.
Why scanning it wouldn't have helped much
These bugs are semantic, not syntactic. There's no pattern that matches "this query should have filtered by user ID and doesn't", because the code looks identical to code that's correct. Rule-based scanners match against known-bad patterns, and an absence isn't a pattern.
We built a benchmark partly to find out how bad this is. RealVuln scores 26 scanners against 66 Python repositories, with 1,903 vulnerabilities labelled by hand and 279 deliberate false-positive traps mixed in. A three-tier ordering falls out and holds under every metric we tried. Security-specialised systems reach F3 86.5 at 0.89 recall. The best general-purpose LLM, Claude Opus 5, manages 67.7. The best rule-based tool, SonarQube, gets 14.4, at 0.063 recall.
That last number is worth sitting with. SonarQube's precision is reasonable at 0.611. It just doesn't find much. Semgrep does better at F3 19.4 and is still an order of magnitude behind the top tier.
Two caveats, since we're the ones publishing this and Kolega scores at the top. Only two rule-based tools were in the run, so "rule-based tools score badly" rests on a sample of two. And the repositories are public, so some may sit in the training data of the LLMs we tested. The false-positive traps exist partly to test discrimination rather than recall, and v2 is adding repositories published after model cutoffs. Every label, scanner output and scoring script is released so you can check rather than take our word for it.
One caveat we should be upfront about: RealVuln's repositories are human-written. It doesn't measure how scanners perform on AI-generated code specifically. Our suspicion is that detection doesn't generalise cleanly across authorship, but we haven't shown that yet, and we're extending the benchmark to find out.
What actually helps
Fix the scaffolding once. Most of those 561 findings trace back to a handful of template decisions, and correcting CORS, credential handling and rate limiting at that level clears the majority before a single feature gets written.
Scan everything rather than a sample. If review has stopped covering what gets generated, whatever replaces it has to run against all of it.
Then the part people skip: 23 findings per app is a backlog, not a fix. That's the bit Kolega works on. Following dataflow through the codebase rather than matching patterns, and attaching a fix to each finding rather than a ticket.
AI didn't make code insecure. It widened the gap between writing code and checking it, past the point any team covers by hand.
Links to add on publish
RealVuln dashboard,
https://realvuln.kolega.dev/Blog: Vibe Coded & Vulnerable (methodology)
Glossary: BOLA
Glossary: AI-Generated Code Vulnerabilities
Up-link:
/use-cases/ai-generated-code-security