Validate AI Outputs on Real Inputs Before Shipping FeaturesValidate AI Outputs on Real Inputs Before Shipping Features
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Most AI features get approved after one good demo. Then real users bring inputs nobody tried, and the AI gives answers that sound right but aren't.
Before any screens get built, I test the feature on 20 real examples.
1. Collect 20 real inputs. 2. Write what a good answer must contain, and what it must never do. 3. Set the pass bar before you look. 4. Score every output as pass, fix or fail. 5. Run the same 20 again after every prompt or model change.
On LeanDeck, a prompt change that fixed one case could break another, and nothing told us. Now the test set comes first.
How do you check AI output before it ships?
Post image
Vansita's avatar
Testing real inputs early makes sense.
Iftiak's avatar
Thanks Vansita. Do you test with real inputs on your projects too, or mostly after launch?
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started