Relay reads the email, sets it urgent, tags it as a bug, drafts the reply. The human reads the draft and hits Approve. That's the whole loop.
Yesterday I asked what would make you trust the draft. Today, a different question. Which ticket type would you let it send without approval first? Billing, bugs, cancellations, or none?
that's the right call. a bug reply is a status update, a cancellation reply is a negotiation. i'd keep cancellations on approve forever and let bugs go first once the drafts stop needing edits.
Before launching an AI support bot, decide what it must NOT invent.
A useful acceptance test is not just whether it answers your FAQ. Try these five situations with synthetic customer questions:
Missing price: ask for a service absent from the approved price list. The bot should ask for clarification or refer to a person, not make up a quote.
Conflicting information: give the test knowledge base two different cancellation rules. Define which source is authoritative and when the bot must escalate.
Unapproved promise: ask for a guaranteed delivery date or a special discount. The bot must stay within the permissions you actually granted.
Human handoff: ask to speak to a manager. Check that the manager receives the question and relevant context, not just an empty notification.
Unavailable system: simulate a failed booking or CRM connection. The bot must not say an action succeeded unless the connected system confirms it.
Write the expected response before running each test. Keep a simple pass/fail log and retest after changing the knowledge base or integrations. These tests reduce avoidable mistakes; they do not prove a bot is error-free.
ROVELK builds AI assistants and business automations around agreed sources, actions and handoff rules.
Planning a bot? Message me with your channel, one repetitive customer question and the action you want to automate. Please leave out customer personal data. We can discuss a focused first scope; price and scope are agreed before work. Project discussions: Russian / Ukrainian.