API smoke QA works best when the first pass covers contract boundaries: auth failure, missing fields, invalid IDs, duplicate requests, rate-limit behavior, and error payloads. I document request and response evidence and prioritize defects by release risk. Available for a 24-hour focused API smoke review: https://contra.com/s/c2YLMmkm-24-hour-api-smoke-test-and-defect-report
Your AI or automation passed every test. Is it ready to ship?
In October I ran 9 of my own projects live, on real data. 9 real bugs showed up after every automated test had passed. So I wrote down the 12 tests I now run before anything ships, and made them a free 3-page PDF.
Three of them:
1. Run it live, not only on saved pages.
A price tracker passed every test, yet every live page came back empty. It asked for compressed pages it couldn't decode.
2. "Talk to a person" works in every language.
In Spanish, the handoff to a human worked. In English, "I want to talk to a person" got "Type human" back.
3. The same message twice gets one reply.
Webhooks are retried. Without this check, a customer gets two answers or two bookings.
The other 9 cover empty results, dashboards, AI quotes, security, rate limits and the client's own machine.
Use it before you accept a delivery from any developer, including me.
Edge cases found in QA often turn into additional work nobody planned for.
Before planning a feature, ask:
→ What slows the user down?
→ Where does the system block them?
→ What if it fails to load?
→ What if it returns no data?
You’re usually able to find more decisions that need to be made by asking.
The four questions you listed are so simple yet so easy to skip in planning. That flowchart mapping out the no-access and error paths is a great artifact, it makes the invisible work visible.
Lead QA for MyFitnessPal’s iOS and Android apps. The project had no specs, so I built all test documentation from scratch — checklists, test cases, test plans — and caught critical crash bugs on both platforms before release.