Review AI-built software before production

Is your AI-built application about to handle real users or production data? The Production Readiness Audit establishes a prioritised baseline. If your team continues developing with AI afterwards, the AI Code Review Retainer can review relevant changes on an ongoing basis.

An application can appear complete in demo mode and still be unready for production. This applies to software from Lovable, Cursor, Claude Code, Bolt, Replit, v0 and other AI-assisted development paths just as it applies to conventionally developed code.

The difference becomes visible once real users, separate tenants, customer data, payments or binding availability requirements are introduced. At that point, the intended path is not enough. Unauthorised paths, failures, misconfiguration and recovery must also be controlled.

1. Secure the repository and baseline

The assessment starts with a complete and reproducible baseline, not with a scanner.

Clarify:

• Is the complete code held in a repository?
• Can the application be built outside the original AI builder?
• Are lockfiles, runtime versions and start commands available?
• Which branches and deployments are actually in production?
• Are generated files or manual platform changes missing from Git?
• Which known defects, workarounds and open prompts affect the current state?

A repository is a reliable source only if build and deployment can be reproduced from it. If the application runs only inside the builder workspace or through undocumented changes in a platform interface, takeover readiness is already unclear.

The audit should refer to a named commit or tag so findings remain attributable to a specific version.

2. Model system and data paths

AI-built code is often created component by component. Production requires a system model that connects those components.

The model should include at least:

• browsers, mobile clients and administrative interfaces
• frontend, backend, API routes and server functions
• databases, object storage and caches
• authentication and identity providers
• background jobs, queues, scheduled tasks and webhooks
• payment providers, email, analytics and other external services
• LLM APIs, vector databases and RAG data paths
• staging, production and local environments

For every data path, document source, destination, permission and data class. This matters particularly where a frontend accesses Supabase or Firebase directly. In that architecture, essential security boundaries sit in RLS, storage policies, claims and service-side roles rather than only in visible application code.

A reliable review also asks which data is sent to external AI services, logging providers or error-analysis tools. Technical telemetry can unexpectedly contain user data, prompts or identifiers.

3. Assess authentication and authorisation separately

Working sign-in only demonstrates that an identity is recognised. It does not prove that the identity can perform only authorised actions.

The assessment needs real roles and negative user journeys:

1. Can a standard user call administrative endpoints directly?
2. Can an object identifier be changed to read or modify another user's records?
3. Do RLS rules isolate tenants during direct API access?
4. Are storage buckets and download URLs protected appropriately?
5. Are roles checked server-side or merely hidden in the frontend?
6. What happens to sessions after a password change, role removal or deactivation?
7. Are invitations, password reset and OAuth redirects constrained?

Multi-tenant products require at least two test tenants and multiple roles. Otherwise, it is impossible to verify that access control works outside the expected happy path.

OWASP ASVS provides a structured verification basis. The relevant requirements depend on application and risk, not simply on the use of AI during development.

4. Assess dependencies, CVEs and secrets

AI tools add libraries quickly. This can lead to duplicate packages, unnecessary SDKs, outdated examples or dependencies whose role no one on the team can explain.

The inventory includes:

• direct and transitive dependencies
• lockfiles and versions actually installed
• known CVEs and vendor advisories
• EOL versions of runtimes, frameworks and database drivers
• unused or replaceable packages
• container base images and operating-system packages
• secrets in Git history, configuration and build logs

A CVE list is not yet a priority list. Deployed version, reachable code path, exposure, existing controls and possible impact determine the required action. A critical CVSS score can sit in an unused function, while a lower-rated weakness may still be urgent in a public business workflow.

Exposed secrets must not only be removed from the current file. They need revocation or rotation because they may remain in Git history, logs or artefacts.

5. Test business logic and interfaces

Static analysis sees only part of production risk. Flows that change money, permissions or durable data deserve particular attention.

Typical review areas include:

• server-side validation of all external input
• idempotency of webhooks and payment events
• signature verification for incoming webhooks
• upload size, file type, filename and malware risk
• race conditions around bookings, quotas or status changes
• error handling and retry behaviour for background jobs
• timeout, rate limit and cost controls for external APIs
• prompt injection and tool permissions in AI features
• safe administrative bulk actions and exports

Critical user journeys need traceable tests. This does not necessarily mean hundreds of unit tests immediately. Sign-in, role changes, data access, payment, export, deletion and core business rules should be verifiable before every release.

6. Secure CI/CD, staging, production and rollback

A production system should not depend on one person pressing the correct button in an AI builder.

A reliable delivery path separates:

• development, staging and production values
• secrets and service accounts by environment
• database migrations from application deployment
• automated checks from deliberate approvals
• preview environments from production data

Every release needs an identifiable version, visible test results and a rollback path. Rollback does not always mean starting the previous image. Non-backwards-compatible database migrations, new queue messages or changed storage structures require their own recovery plan.

The team should also test whether the system can be rebuilt from repository, configuration and backups. A deployment that works only on one accumulated environment is not reproducible.

7. Validate logging, monitoring, backup and restore

Production readiness does not end with a successful deployment.

The operational review asks:

• Which technical errors and business events are logged?
• Are secrets and sensitive data kept out of logs?
• Which key functions are actively monitored?
• Who receives an alert, and who responds?
• Which data and configurations are backed up?
• How long are backups retained?
• Has restoration actually been tested?
• Which outage and data-loss objectives apply?

A backup without a restore test is an assumption. A documented sample test may be sufficient for a small application. Business-critical systems need recurring evidence aligned with their recovery objectives.

Monitoring also needs to test more than the home page. An HTTP 200 response says little about whether sign-in, database, jobs, email or storage work.

8. Prioritise, remediate and retest findings

An audit should not produce an unordered scanner wall. At least four categories are useful:

Every finding needs an affected component, a traceable risk, a recommended measure and an owner. Critical fixes are retested. Only then can a meaningful go-live decision be made.

The audit itself does not automatically remediate issues. If WZ-IT should implement them, software maintenance or ongoing development can be added as a separate allowance. Infrastructure, monitoring, backups and incident response are scoped through Managed Operations.

Keep developing with AI after go-live

Controlled development does not mean banning AI tools. Teams can continue using prompts, agents and code assistants. From this point, however, the delivery path should be traceable:

1. Changes land in the repository.
2. Automated tests and scanners provide technical signals.
3. Risk-relevant changes receive human review.
4. Approved versions are deployed reproducibly to staging.
5. Critical user journeys are tested.
6. Production is updated and observed in a controlled process.

For active products, a monthly or weekly review cadence can keep the baseline from becoming obsolete within a few releases. Implementation and operations remain separate responsibilities even where they are supplied by the same provider.

Sources

• NIST Secure Software Development Framework, SP 800-218
• OWASP Application Security Verification Standard
• OWASP Top 10
• GitHub: About secret scanning
• UK NCSC: The vibe coding spectrum