QA for AI Customer Support Drafts: Catch Hallucinations Before You Send

AI customer support needs different quality checks for email drafts and direct chat answers, because fluent replies can still invent policies, steps, or facts. Large language models produce confident prose regardless of truth. In a support reply, a polished sentence easily disguises an invented refund window, a non-existent settings toggle, or an impossible workaround.

For email drafts, a person can check those claims before sending; direct AI chat answers reach visitors without that review. There There (there-there.app), the AI helpdesk built by Spatie in Antwerp, Belgium, uses AI drafts for email and direct AI answers in the chat widget. This guide covers email draft review and the limits of chat safeguards, with one principle throughout: AI for routine questions, people for hard cases such as upset customers, refunds, and edge cases.

Key takeaways

• AI email drafts can sound right while being factually wrong, so a person must check claims against the team's knowledge before sending.
• In the chat widget, AI answers visitors directly from the team's own knowledge and past tickets, and every AI answer offers a way to reach a person.
• Grounding does not eliminate hallucinations; use AI for routine questions and involve people in upset customers' concerns, refunds, and edge cases.

What AI customer support means for email and chat

AI customer support can mean drafting a reply for an agent or answering a visitor directly. The difference determines whether a person can check an answer before the customer sees it.

For email in There There, AI drafts replies on tickets using the team's knowledge and past tickets. A person reviews the email draft, edits it, and sends the reply. If you want a full exploration of the category itself, our guide to the AI helpdesk covers that broader ground.

In the chat widget, an AI answer goes directly to the visitor without human review first. An incorrect email draft can be caught before sending, but an incorrect chat answer may reach a visitor before a person gets involved. Email therefore needs a pre-send review habit; chat needs maintained knowledge and a clear route to a person.

How AI support drafts go wrong (hallucinations in practice)

An AI hallucination in customer support rarely looks like gibberish. If an AI generated broken syntax, any specialist would discard it immediately. The real hazard is the fluent hallucination: a response that reads naturally, adopts a warm tone, and is completely wrong on the facts.

In daily support queues, these errors consistently fall into four patterns:

1. Invented policies: A customer asks for an exception, and the draft cheerfully promises a sixty-day return window even though your policy limits refunds to fourteen days.
2. Fabricated product steps: A user asks how to export data, and the draft invents an intuitive “Settings > Advanced Export” tab that does not exist in your software.
3. Stale facts and limits: The model cites legacy pricing or retired plan limits found in an outdated ticket thread.
4. Confident false negatives: A user asks about a feature you launched last week, but the model cannot find it and claims your platform does not support it.

These failures come from ungrounded generation, stale documentation, and retrieval gaps where the system misses relevant context. Effective QA is not about editing grammar. For email drafts, it means checking factual claims before sending. For direct chat answers, it also means maintaining the knowledge AI uses and correcting errors when they are found.

A practical QA checklist before you send an email reply

Because language models write with unearned confidence, support teams need an explicit operating routine for reviewing drafts. This is an operating checklist for your team, not a software audit module. Treat every AI draft as an unverified suggestion from an eager colleague who has skimmed your documentation but has never used the product.

Check factual claims against knowledge

Every specific noun, number, interface path, and policy condition in an AI draft is a factual claim requiring verification. Scan past the conversational filler and isolate the core statements:

• Numbers, timeframes, and limits: Confirm that quoted export times, seat limits, or trial durations match current documentation.
• Product navigation: Verify that menu items, button labels, and settings tabs actually exist in the live interface.
• Policy conditions: Ensure rules for refunds, cancellations, and data ownership reflect active policies rather than an inferred compromise.

If a draft contains a claim you cannot point to in your knowledge base or verified tickets, delete it. Never approve a statement simply because it sounds plausible.

Flag unknowns instead of guessing

Language models predict the next word in a sequence. When presented with a question your knowledge base does not cover, a model will rarely pause to say it lacks information. Instead, it improvises details to complete the reply.

In human-led customer support, admitting uncertainty is a sign of competence. Train your team to treat an escalation or an admission of an unknown as a success. If a draft guesses at an edge case, delete the speculation and check with product or engineering before replying. Customers appreciate honesty; they rarely forgive confident misinformation.

Match tone and commitment level

Drafting tools frequently lean toward excessive deference and premature commitment, making operational promises your company cannot honor.

Watch out for unauthorized commitments:

• “We will have this bug fixed by tomorrow morning.”
• “I will ensure our product team adds this feature next sprint.”
• “I have applied an exception to your billing profile.”

An AI draft cannot assess engineering bandwidth or contractual liability. The human reviewer must check the commitment level, stripping out unapproved promises and framing replies around what your team can legitimately deliver.

Keep high-stakes cases with people

Certain customer conversations carry financial, security, or legal consequences that make quick approvals dangerous. These threads require deliberate, line-by-line review.

For email tickets and chat conversations handed over to the team, require full manual review or a fresh reply for these cases:

• Billing discrepancies, chargebacks, and refund requests
• Suspected account compromises and security inquiries
• Legal notices, contractual disputes, and data deletion requests under privacy regulations
• Cancellation requests from distressed or angry customers

Using AI to summarize a thread or draft an internal note saves time, but sending an AI draft without careful verification introduces unacceptable risk. When the stakes are high, human judgment must lead the conversation. In the chat widget, every AI answer offers a way to reach a person; this is a handoff route, not a human approval step before the AI answers.

Knowledge hygiene that makes drafts safer

An AI drafting assistant reflects the state of your team documentation. If your internal documentation is fragmented, contradictory, or neglected, the drafts your team receives will inherit those flaws. Conflicting guidance makes both human review and direct AI answers less reliable.

Safer drafts begin with active knowledge hygiene. Support leads must treat internal documentation as an active operational asset rather than a completed project. When a product team updates an interface, changes an export format, or revises a billing model, the corresponding internal articles must be updated immediately. Deprecated guides should be archived or deleted so the retrieval system cannot serve them as reference context.

Past resolved tickets are also helpful reference material, but they require strict curation. Support teams often solve problems using temporary workarounds while waiting for a patch. If those historical tickets remain in the retrieval pool indefinitely, a drafting tool might suggest a complex workaround for a bug resolved long ago. Periodically reviewing which ticket categories feed into your drafting context prevents historical edge cases from polluting current replies.

When your team maintains clean, non-conflicting reference material, drafts become significantly more accurate. For a detailed breakdown of structuring team documentation for both humans and AI, see our guide on the knowledge base for customer support.

Where automations help (and where they don’t)

Support teams adopting AI often confuse deterministic automations with generative language models. Automations and AI drafting tools solve different operational challenges, and understanding the boundary between them is critical for quality control.

Automations operate on fixed logic: triggers, conditions, and actions. They excel at repetitive, deterministic operations that demand absolute consistency. An automation can inspect an incoming ticket, detect an enterprise email domain, assign the ticket to a senior specialist, and tag it as an urgent priority. It can also bring relevant customer account data into the sidebar so the reviewer has immediate context. By putting the draft and the necessary background information in front of the right person, automations clean up the review process.

Routing and tagging do not verify the factual accuracy of a reply. Do not treat those automations as evidence that an answer is safe to send.

Automations should support the human reviewer by handling routing, tagging, and contextual preparation. For email drafts, the specialist checks factual accuracy before sending..

How There There handles email and chat differently

There There's approach is AI for routine questions and people for the hard cases. How an answer reaches the customer depends on the channel.

On email tickets, There There uses AI only to draft replies. A person edits the draft and sends the email reply. That gives the person a chance to catch mistakes before the reply reaches the customer, although human review can still miss errors.

In new chat widgets, AI is on by default and answers visitors directly, without human review before the answer goes out. AI chat is available on paid plans only. The AI answers using only the team's own knowledge, including docs, help articles, connected sources, and past tickets. Every AI answer offers a way to reach a person. When a visitor asks for one, There There creates a ticket with the full transcript, and the team replies in the chat widget or by email.

These chat safeguards are answers based on the team's own knowledge and an easy, always-offered route to a person. They reduce risk, but they do not guarantee that a chat answer is correct or stop every hallucination from reaching a visitor. Use AI for routine questions and bring people into conversations involving upset customers, refunds, or edge cases. The chat widget currently does not tell visitors they are talking to AI before the first answer. For more on the balance between routine help and human judgment, see our customer service philosophy guide.

Common pitfalls

When teams introduce AI into their support operations, they often fall into predictable habits that undermine answer quality. Being aware of these pitfalls helps safeguard standards as ticket volume grows:

• Measuring speed while ignoring accuracy: Optimizing purely for first response time encourages agents to skim AI drafts and hit send without verifying facts. Delivering an invented policy or incorrect troubleshooting step creates more work for your team and frustrates the customer. For a deeper discussion on balancing resolution speed with quality, read our analysis of first contact resolution.
• Assuming grounded drafts are infallible: Grounding an AI model on documentation can reduce hallucinations, but it does not make the output immune to error. If your documentation contains subtle ambiguities, the model can still draw incorrect inferences. For email, check the draft before sending; for chat, do not assume a knowledge-based answer is correct simply because it uses your own sources.
• Treating direct AI answers as a substitute for people: AI can help with routine questions, but upset customers, refunds, and edge cases need human judgment. Keep the route to a person easy to use, and take responsibility when an AI answer is wrong.
• Expecting software to audit software: Relying on automated evaluation tools to check AI drafts creates a chain of unverified assumptions. No statistical model understands customer relationships, product subtleties, or company principles as well as a trained support specialist.

FAQ

How does AI customer support differ between email and chat?

In There There, AI drafts replies on email tickets, and a person edits and sends them. In the chat widget, AI answers visitors directly from the team's own knowledge and past tickets. Every AI chat answer offers a way to reach a person; asking for one creates a ticket with the full transcript.

What is an AI hallucination in a support reply?

An AI hallucination is a generated statement that is factually incorrect, ungrounded, or fabricated, presented in a convincing tone. In support, hallucinations usually appear as non-existent product settings, incorrect return terms, inaccurate pricing, or invented technical workarounds.

Does grounding on knowledge prevent hallucinations?

No. Grounding a language model on internal docs and resolved tickets reduces errors, but models can still misinterpret phrasing, retrieve the wrong section, or rely on conflicting historical tickets. Review email drafts before sending. Chat answers go out without that review, so maintained knowledge and a route to a person matter, but neither guarantees accuracy.

How is this different from an AI helpdesk overview?

This guide focuses on reviewing AI email drafts and understanding the safeguards and limits of direct AI chat answers.

Can small teams QA drafts without a dedicated QA product?

Yes. Effective draft QA is an operational discipline, not an expensive software suite. For email, agents can check claims against documentation and remove speculative guesses before sending. For chat, teams must maintain the knowledge AI uses and be ready to take over when a visitor asks for a person.

Where do routing automations fit?

Automations handle deterministic tasks such as assigning tickets, tagging inquiries, and attaching customer context based on triggers and conditions. They ensure drafts reach the right specialist for review, but they do not evaluate factual accuracy.