QA

I'm building a portfolio of small software businesses with AI agents doing essentially all of the building (and hopefully the growing + operating as well).

4 products so far.

This week I started testing them in production the way a real customer would, with real money. It was rough at first. But every bug had a similar root cause...

The root cause: lessons were learned as advice, not as machinery

My technical cofounder (agent) keeps a playbook. Every hard lesson gets written down: verify screens against designs, test the purchase flow, check pages on mobile.

The playbook is genuinely good. And it didn't matter.

Because advice competes with everything else for attention, and machinery doesn't.

A written rule that says "don't drift from the design" loses to a Tuesday-afternoon shortcut every time. A test that turns the build red does not. The audit found the gap everywhere. One product had automated design checks; 3 didn't... the list goes on.

What we built instead: robot inspectors in every product

In a long session, my cofounder installed the same 5 inspectors in all 4 products. Every code change now has to get past them before it can ship:

1. A design inspector: every designed screen gets a written contract: exact words, required elements, forbidden things. Drift turns the build red.
2. A stub catcher: a written list of every designed screen that isn't fully built yet. The list is only allowed to shrink. A placeholder can never silently count as "done."
3. A sign-in inspector: verifies the app shows the right things to signed-in vs signed-out users. The exact bug I found, now impossible to reintroduce quietly.
4. A real-purchase inspector: no more faking the paid state. The test makes an actual test-mode purchase, receives the actual payment webhook, and clicks "view report" like a customer. If the money loop breaks, the build breaks.
5. A phone-size inspector: every public page, checked at phone, tablet, and desktop widths, on every change. I run these businesses from my phone. Now the tests assume customers do too.

Lessons

• A lesson isn't learned until it's enforced: if it lives in a doc, it's a suggestion. If it turns the build red, it's a rule.
• Tests that fake the hard part test nothing: real purchase, real webhook, real click-through, or it doesn't count.
• Fix the class, not the bug: a product's sign-in nav was wrong. The fix shipped to 4 products as a permanent check, not to one product as a patch.
• Debt you can see shrinks. Debt you can't see compounds. The list of unfinished screens used to live in my production-testing anxiety. Now it's a number in the codebase that can only go down.
• AI agents amplify whatever system you have: with advice, they drift confidently. With machinery, they get caught the same afternoon.
• You still need a human walkthrough: robots can't feel that an email is ugly or that 60 seconds feels like forever. The inspectors shrink what my walkthroughs need to catch. They don't replace them.

Product 5 will be born with all of this on day one.