How to Set Up Escalation Rules So Hard Questions Reach Your Team

There is a number every chatbot vendor puts on the front page: containment rate, the share of conversations the bot handles without a human. It is easy to understand, easy to benchmark, and easy to raise by making it harder to reach a person.

That last property is the problem. A conversation where the customer asked three times for a human, gave up, and closed the window counts as contained. So does a conversation where the bot answered confidently and wrongly. Containment measures conversations that ended, not problems that got solved, and the gap between those two is where support programmes quietly go bad.

This guide is about the other half of the design: deciding which conversations should leave the bot, making that route fast and unembarrassing, and giving the human on the other end enough context to be useful on arrival.

What the Benchmarks Actually Say

Containment ranges vary widely by industry, and the variation is structural rather than a matter of vendor quality.

The pattern is that containment rises with question uniformity. E-commerce sits at the top because "where is my order" is the same question ten thousand times with a different order number. Healthcare sits at the bottom because the questions are varied, consequential, and often regulated. If your bot is running at 45 percent in a SaaS business, that is inside the normal band, and chasing 80 percent because a retail case study mentioned it is chasing the wrong target.

Two further reference points are worth holding. Gartner's 2025 benchmark for a well-configured retrieval-based chatbot is 40 to 65 percent containment, and most deployments start between 20 and 40 percent and improve from there. Mature programmes reach 70 to 90 percent, but they get there by expanding what the bot legitimately knows, not by narrowing the exits.

Containment Without Resolution Is a Failure Wearing a Success Costume

The single most useful discipline in this area is refusing to look at containment on its own. Pair it with a resolution measure, and the picture changes.

The second row is the common one, and the tell is easy to look for: conversations that end without the customer confirming anything, with a final message from the bot and no reply. Sample fifty of those a month and read them. It takes twenty minutes and it is the most honest measurement available.

The Escalation Triggers Worth Wiring In

Escalation should not be a single "I do not know" fallback. It is a set of independent conditions, each catching a different failure. Seven cover most of what happens in real traffic.

1. The customer asks. A direct request for a human escalates immediately. No confirmation prompt, no "are you sure", no attempt to answer once more first. This is the one rule with no exceptions, and it is the one most often violated by teams optimising containment. Roughly nine in ten consumers say a person should be available if they decline to deal with AI, and making them fight for it converts a neutral interaction into a complaint.

2. Low retrieval confidence. If the best matching passage is weakly similar to the question, the bot does not have the answer. Escalating here is strictly better than generating, because the alternative is the confident fabrication problem in its most expensive form.

3. Negative sentiment. Profanity, all caps, threats to cancel, "this is ridiculous". Sentiment is one of the strongest available triggers precisely because it fires before the customer has explicitly asked for anything, and it is what separates a recoverable interaction from a churn event.

4. Repetition and loops. The customer rephrasing the same question a third time is a stuck conversation regardless of what the confidence score says. Loop detection catches the case where retrieval keeps returning something plausible and useless.

5. Out-of-scope subject matter. Legal, medical, tax, contractual negotiation, security incidents, anything touching an individual account balance. These escalate on subject alone, before any attempt at an answer.

6. High-value or high-risk context. An enterprise account, an active churn signal, a customer in a cancellation flow. If your chat widget knows who it is talking to, the threshold for escalation should not be uniform across everyone.

7. Explicit business events. A refund above a threshold, a complaint that names a regulator, a mention of legal action. These are cheap to detect on keywords and expensive to miss.

Each trigger should be independently switchable and independently measurable. When escalation volume spikes you want to know which rule fired, not just that something did.

The Handoff Is the Part Everyone Gets Wrong

Getting the customer out of the bot is the easy half. What happens next is where the experience is actually won or lost, and the industry record here is poor. Reporting on AI-to-human handoffs finds that only about 15 percent of consumers experience a seamless transition, and Cisco research found that roughly a third of agents receiving an escalated conversation lack enough context to help.

That produces the single most infuriating moment in customer service: repeating your whole problem to the second party who was supposed to already have it.

A handoff that works carries five things across:

• The full transcript, not a summary alone. The agent should be able to read what the bot actually said, because sometimes the bot said something wrong and the agent needs to know what they are correcting.
• A structured summary at the top. What the customer wants, what has been tried, what was ruled out, what the bot could not confirm. This is the difference between a two-minute and a ten-minute resolution.
• The identity and account context the bot already collected, so nobody asks for an email address twice.
• The reason for escalation. Which trigger fired. "Customer asked for a human" and "confidence below threshold" call for different opening lines.
• A visible state change for the customer. They should be told a person is joining, roughly how long it will take, and what happens if nobody is available.

That last point is what turns a handoff from an abandonment risk into a wait. The mechanics of a good handover matter less than the customer's certainty that one is happening.

What Happens When Nobody Is There

Escalation rules assume an agent exists. At 2am, on a small team, that assumption fails, and the design that handles it badly is worse than having no escalation at all, because you have promised a human and not delivered one.

Three workable patterns, in order of preference:

1. Scheduled availability with honest messaging. The bot knows business hours and says so: a person will pick this up at 9am, here is your reference, here is your email confirmation. The customer's expectation is set correctly and nothing is broken.
2. Asynchronous capture. Collect the question, the transcript, and contact details, and create a ticket. The conversation continues by email. This is the right default for most small teams and it converts an out-of-hours escalation into a normal support queue item.
3. Callback or scheduling offer. For high-value contexts, offer a booked slot rather than an open-ended wait.

What does not work is a handover request that silently goes nowhere, or a bot that keeps trying to answer after the customer has asked for a person. Both are recoverable failures that get treated as unrecoverable by the customer.

Setting the Threshold: The Trade Nobody Escapes

Every escalation rule has a false positive rate and a false negative rate, and they move in opposite directions. Tighten the confidence threshold and you escalate conversations the bot could have handled, which costs agent time. Loosen it and you contain conversations the bot got wrong, which costs trust.

The asymmetry is worth thinking through rather than splitting the difference. A wasted escalation costs a few minutes of agent time. A contained wrong answer can cost a customer, a chargeback, or, in the Air Canada case, a tribunal ruling. Unless your support team is at capacity, the expected cost of over-escalating is lower than the expected cost of under-escalating, and thresholds should be set accordingly.

There is one exception worth naming. If escalation volume exceeds what the team can answer within its stated response time, over-escalating stops being cheap, because the queue itself becomes the bad experience. At that point the fix is upstream: work out which escalations were avoidable, and fix the content gaps that caused them.

Escalation Data Is Your Content Roadmap

Every escalation is a documented instance of a customer needing something the bot could not provide. Treated as a queue to clear, it is a cost. Treated as a dataset, it is the most precisely targeted content backlog you will ever have.

A workable weekly routine:

1. Export escalations from the last seven days, tagged by trigger.
2. Sort by trigger. Confidence-threshold escalations are usually content gaps. Sentiment escalations are usually product or process problems. Direct human requests are often trust problems, and worth reading closely for a pattern.
3. Cluster the confidence escalations by topic. Three or more on the same subject means a missing document, and writing it removes the whole cluster.
4. Write one document. Not five, one, addressing the largest cluster in the customer's own phrasing.
5. Re-measure the following week.

This is the loop that raises containment legitimately, and the difference matters. Containment that rises because the bot learned something is durable. Containment that rises because the exit got harder to find reverses the moment a customer writes a review about it. The same logic underpins the broader case for designing the handover path deliberately rather than treating it as a failure state.

A Practical Starting Configuration

For a team deploying escalation for the first time, this configuration is defensible on day one and refinable later.

Start with the first three. Add the rest as you see which failures actually occur in your traffic, and instrument each one so you can tell them apart in the data.

Where the Platform Fits

Escalation is one of the areas where the tooling genuinely determines whether the design is achievable. Rules are easy to write and hard to run if the bot and the human inbox are separate systems: the transcript does not travel, the mode switch is manual, and nobody can tell which rule fired. Paperchat handles conversations in one place, so a conversation can move from AI mode to staff mode with the full history intact, escalation is a state on the conversation rather than a copy into another tool, and the transcripts that follow are available for the weekly review that turns escalations into content.

The Bottom Line

Containment is a fine metric and a terrible target. Raise it by teaching the bot more, never by making the exit harder to find, and always read it next to a resolution measure so you can tell a solved problem from an abandoned one.

Escalate on seven independent conditions rather than one fallback: the customer asking, low confidence, negative sentiment, loops, out-of-scope subjects, high-value context, and defined business events. Carry the transcript, a structured summary, the identity, and the escalation reason across the handoff, and tell the customer what is happening. Handle the no-agent case honestly instead of pretending. Then read last week's escalations as a content backlog, because the ones that repeat are telling you exactly what to write next.