Somewhere between the model's confident output and the production deploy, a question often goes unasked: was any of this actually reviewed for security? AI coding tools have quietly changed what it means to ship software, and most security processes haven't caught up.
GK
Gauri Khatate
โ๏ธ Security Researcher & Technical Writerยท๐ 5 min read
The conversation around AI in software development has mostly been about productivity. Faster feature delivery, reduced boilerplate, fewer blank-screen moments. All of that is real. What gets less airtime is the security dimension, not because it is a niche concern, but because it does not surface immediately. The problems tend to arrive later, when an audit surfaces a pattern nobody noticed, or when an incident trace leads back to a function nobody wrote by hand.
AI coding assistants generate code from patterns learned across billions of lines of public source code, including the insecure ones.
How AI Coding Tools Actually Work, and Why That Matters
AI coding assistants, GitHub Copilot, Cursor, Amazon CodeWhisperer, generate code by predicting what comes next based on patterns learned from billions of lines of publicly available source code. That is a remarkable capability. It is also the source of the core security tension.
Public code is not a security benchmark. It contains deprecated patterns, insecure defaults, hardcoded credentials used in tutorials, and architectural shortcuts that made sense in context but should not be generalised. The model does not distinguish between code that was written securely and code that just happened to be widely used. It learns from both equally.
โThe model doesn't know the threat model. It doesn't know who's calling the endpoint, what data is flowing through it, or what the blast radius looks like if the validation is wrong.โ
Five Security Risks That Show Up Consistently
The vulnerabilities in AI-generated code are not random. They cluster around predictable patterns, which makes them knowable, and in most cases, preventable.
01
Insecure Defaults Reproduced at Scale
AI models reproduce the patterns they were trained on. A significant portion of public code uses insecure defaults, disabled certificate validation, wildcard CORS headers, debug logging left on, error responses that expose stack traces. These patterns get reproduced confidently and at speed, with no visual cue that the default is wrong.
02
Stale Dependencies with Known CVEs
When an AI tool suggests an import or recommends a library, it is drawing on training data with a cutoff date. That library may have accumulated critical vulnerabilities since the model learned about it. Automated dependency scanning is the only reliable check, no human reviewer can carry the current CVE list for every package in the ecosystem.
03
Shallow Input Validation
Generated code tends to optimise for the expected case. Malformed input, boundary conditions, type coercion edge cases, and injection vectors often get handled incompletely, or not at all. In any API surface or user-facing feature, this is a direct path to SQL injection, XSS, and logic bypass vulnerabilities.
04
Credentials That Make It to Version Control
AI tools generate working examples. Working examples often include credentials, tokens, or API keys as placeholders that get committed without replacement, or used in development environments and never cleaned up before the branch lands in main. Secrets scanning on every commit catches what code review misses.
05
Over-Permissioned Infrastructure Code
AI-generated Terraform, CloudFormation, and IAM configurations tend toward permissiveness because permissive configurations are easier to demonstrate working. A scaffolded IAM role with admin access works in every example. A role scoped to minimum required permissions requires context the model does not have. Least privilege must be applied on top of AI output, it does not emerge from it.
Automated static analysis on every commit is the only scalable check when AI-generated output is entering the codebase at volume.
The Velocity Trap
There is a subtler problem that does not get discussed enough: the relationship between AI coding speed and the natural friction that catches security issues. When a developer writes code manually, there are moments of friction that sometimes work in the codebase's favour, reading back through logic before committing, the slight pause before accepting an approach that feels off.
AI-generated code arrives complete. It is formatted, it compiles, the tests often pass. The natural response is to accept it and move on. The friction that catches issues is missing, and with it, a layer of informal security review that most teams did not realise they were relying on.
What This Means for SOC 2 and Compliance Teams
๐ SOC 2 Common Criteria, AI Code Intersections
CC8.1, Change management: Are all changes reviewed before production? Does AI-generated code go through the same rigour as human-written code?
CC6.1, Logical access: Are access patterns in generated infrastructure code validated against least privilege?
CC7.1, Monitoring: Is AI-generated code included in vulnerability scan scope? Are scan results reviewed and acted on?
Auditors are starting to ask about AI-generated code directly. Teams that cannot articulate a clear answer are creating gaps in their control narrative. The expectation is not that AI tools are banned, it is that the review process accounts for what AI tools introduce.
Building the Review Layer That Makes This Safe
๐ The Four Essential Gates for AI-Generated Code
๐
Automated Static Analysis on Every Commit
Snyk Code, Semgrep, and Checkmarx scan for vulnerability patterns before code reaches review. When AI-generated output is entering the codebase at volume, automated scanning is the only scalable check. Manual review catches logic problems, it cannot reliably catch the full surface area that static analysis covers.
๐
Secrets Scanning as a Merge Gate
TruffleHog and GitLeaks scan the full commit history and flag credentials, API keys, and tokens before they land in production. This should be a hard gate, a failed secrets scan blocks the merge. Given how AI tools generate working examples, this category of risk is structurally higher than it was with fully manual code.
๐ฆ
Dependency Scanning on Every Build
Every library introduced through AI tooling should be treated as unverified until scanned. Integrating Snyk Open Source or Dependabot into the CI pipeline means CVEs are caught before deployment, not after. The key is that this runs automatically on every build, not periodically.
โ
An Explicit AI Code Review Checklist
A short checklist, input validation confirmed, no hardcoded values, dependencies scanned, access scope verified, creates a reviewable audit trail and makes the expectation explicit rather than assumed. Teams that have defined what โsecurity reviewedโ means for AI-generated code are in a materially better position than those that leave it to individual discretion.
AI moves fast. Security gaps move faster.
SecComply maps your engineering security posture against SOC 2, ISO 27001, and India's DPDP Act, and shows exactly where AI-generated code is creating compliance blind spots.
Yes. AI models are trained on public code that includes insecure patterns, deprecated libraries, and hardcoded credentials. The model reproduces those patterns without knowing which are unsafe. Research shows 3 in 5 AI code suggestions contain at least one security flaw.
AI-generated code intersects directly with SOC 2 CC8.1 (change management), CC6.1 (logical access), and CC7.1 (monitoring). Auditors now ask whether AI-generated code goes through the same review rigour as human-written code. Teams that cannot articulate a clear answer create gaps in their control narrative.
The four essential layers are: static analysis on every commit (Snyk Code, Semgrep), secrets scanning as a merge gate (TruffleHog, GitLeaks), dependency scanning on every build (Snyk Open Source, Dependabot), and an explicit AI code review checklist to create an auditable trail.
Permissive configurations are easier to demonstrate working. A scaffolded IAM role with AdministratorAccess works in every example. A role scoped to minimum required permissions requires context the model does not have. Least privilege must be applied on top of AI output, it does not emerge from it.
When developers write code manually, friction moments act as informal security review. AI-generated code arrives complete and formatted, removing that friction. The natural response is to accept and move on, bypassing the informal review layer most teams did not realise they were relying on.
