The short answer
SiteVoice began as a prompt-generated web application and worked as a demonstration. Making it safe to ship meant a security assessment that found a row level security vulnerability, a credential rotation process, an end-to-end test suite driving a real browser, and webhook verification. None of that is visible in a demo, and all of it is the difference between something that runs and something you can put users on.
The short version
SiteVoice is a web application for recording and processing audio, originally generated through a prompt-based builder. It worked. Screens rendered, uploads uploaded, the flow completed.
That is what these tools are genuinely good at, and it is worth acknowledging clearly rather than sneering at. Getting from nothing to a working prototype in an afternoon is real value.
What the audit found
Row level security
The most serious finding was a row level security problem. Row level security is the mechanism that stops one user reading another user's rows, and it is exactly the kind of thing a generated application gets wrong, because the generated code produces a working query and the policy behind it is invisible in every test a person is likely to run by hand.
You do not notice it while clicking around as one logged-in user. Everything you can see is yours, so everything looks correct. It only surfaces when a second user exists and somebody checks whether the boundary actually holds.
Credentials
A rotation process was needed and documented. Prototypes accumulate keys quickly, in configuration, in examples, in a commit somebody made in a hurry, and once a credential has been anywhere it should not be, the only real fix is to replace it.
Webhooks
Webhook handling needed verification and a real test pass. An unverified webhook endpoint accepts instructions from anyone who finds the URL, which for a billing or processing webhook means someone else can tell your system what happened.
Tests that drive a real browser
An end-to-end suite was added, running against a real browser with headless and debug modes, plus a mock server so webhook behaviour could be exercised without live traffic.
Prompt-generated applications almost never arrive with tests, because the generation loop optimises for something that looks right rather than something that stays right. The first meaningful change to such a codebase is where that shows: nothing tells you what you broke.
| Area | Before | After |
|---|---|---|
| Data access | Policy assumed correct | Assessed, fixed and documented |
| Credentials | Accumulated in place | Rotation process written down |
| Webhooks | Accepted as received | Verified and tested against a mock |
| Regressions | Found by users | Found by a suite driving a real browser |
What we would tell a founder holding a generated app
Not that it was a mistake. It probably got you to something demonstrable faster than any other route. But treat it as a prototype that reached further than most, and budget for the part it skipped.
- Have the data access rules reviewed before you have real users. Row level security holes are invisible from inside a single account and obvious to anyone who looks properly.
- Rotate every credential once, on the assumption that something leaked into a commit or a screenshot along the way.
- Verify every webhook. An unverified endpoint is an open instruction channel into your system.
- Get one end-to-end test running before you make your first significant change, so you find out what you broke from a test rather than from a user.
Frequently asked questions
- What is a row level security vulnerability?
- It is a failure in the policies that decide which rows a given user may read or write. When it is wrong, one user can reach another user's data. It is invisible while testing as a single logged-in account, because everything you can see legitimately belongs to you.
- Is AI-generated code inherently unsafe?
- Not inherently, but the generation loop optimises for code that works rather than code that is correct under adversarial conditions. Data access policies, credential handling and webhook verification are the areas that reliably need review, because none of them affect whether the demo works.
- Why rotate credentials on a codebase you inherited?
- Because you cannot prove a key has never been exposed. Prototypes accumulate credentials in configuration files, examples, screenshots and hurried commits. Rotating once establishes a known-good starting point that costs an hour.
- Why do webhooks need verification?
- An unverified endpoint accepts instructions from anyone who discovers the URL. For a billing or processing webhook that means an outsider can tell your system that something happened, which is as serious as it sounds.
- Should I rebuild a prompt-generated app from scratch?
- Usually not. The product decisions embedded in it have value and rebuilding discards them. The economical path is normally a security review, a credential rotation, an end-to-end test suite, and then incremental replacement of the weakest parts.
Talking about technical consulting?
Architecture review, technology selection, due diligence and a second opinion when a decision is expensive to reverse.