TL;DR: The hardest part of building an AI receptionist is not the talking. It is deciding when the agent should stop talking and put a human on the line. Every agent I have shipped for hospitals, vet clinics, garages and brokerages has three exits, not one: transfer now, take a message and escalate, or refuse and route. Most bad handoffs I get handed to fix are not transfer failures at all, they are missing exits. Here is how I design the escalation layer, why I write the exits before the happy path, and what happens when nobody picks up.
A voice agent that handles ninety percent of calls and drops the other ten on the floor is not a ninety percent success. The ten percent it drops are almost always the calls that mattered most: the emergency, the angry customer, the deal that was ready to close. Businesses judge the whole system on those calls, and they are right to.
So when I build a voice agent, the escalation path is not a fallback I bolt on at the end. It is part of the spec from the first conversation with the client, and it usually takes longer to get right than the script everyone is excited about.
Three exits, not one
"Transfer to a human" is treated as a single feature. In production it is three different things, and conflating them is what makes agents feel broken.
Exit one: transfer the live call. The caller stays on the line and a person picks up. This is the right exit when the situation is urgent, when the caller has already been through one loop of misunderstanding, or when the business loses real money by making this person call back. It is also the exit with the most ways to fail, because it depends on a human being physically available right now.
Exit two: take a structured message and escalate. The agent captures the details properly, ends the call cleanly, and fires an alert into whatever the team actually watches. This is the correct exit far more often than clients expect, especially outside business hours, and it is the one that gets skipped.
Exit three: refuse and route. The agent declines to handle the topic at all, says so plainly, and directs the caller to the right channel. For a medical practice that is clinical advice. For a brokerage it is anything that touches price negotiation or fair housing. The agent does not need a graceful answer here. It needs a firm one.
Once you have three exits you can stop trying to write an agent that always knows the answer, which is a losing game, and start writing one that always knows where the call goes next.
I write the exit conditions before the script
When I sit down with a client, my first question is not what the agent should say. It is: which calls, if this thing handles them badly, would make you turn it off tomorrow?
The answer to that question is the escalation spec. It is usually short and it is usually specific:
- The vet clinic said any caller describing a collapsed, bleeding or poisoned animal. No triage, no questions, straight to a person. I wrote about that shape in more detail in AI voice agents for veterinary clinics.
- The garage said anyone stranded on a roadside. Whatever they were calling about becomes irrelevant, because a person by a motorway is a different kind of caller.
- The clinic said anything that could be read as symptoms or advice. Not the agent's job at any confidence level, which is the whole argument in AI voice agents for medical clinics.
- The brokerage said any caller who mentions a competing offer. Speed to a human is the entire value of that call.
I build those conditions into the flow as first-class exits before I write a single line of the booking path. It is the same discipline I use when sizing conversation flow nodes and writing exit conditions first: the interesting part of a node is not what it does, it is how the conversation leaves it. It also changes what the client tells you. "Handle our calls" is not a spec. "Never let a stranded driver sit through a booking script" is a spec, and it is testable.
The trigger set that actually catches escalations
Escalation triggers come from four places, and an agent that only uses one of them will miss calls.
Content triggers. The caller says something on the list. Emergency words, legal words, cancellation words, the name of a competitor. These are the obvious ones and they are the easiest to specify, so most agents stop here.
Explicit request. The caller asks for a human. This one is not negotiable and I make it work at every node in the flow, at any point in the call. An agent that keeps steering a caller back into the script after they asked for a person is the fastest way to make someone hate a business. If they ask twice, it should already have happened.
Repair-loop triggers. The agent has failed to understand the same thing twice, or the caller has repeated themselves, or the call has run well past the length this kind of call should take. Nothing was said that hit a keyword. The conversation is just going badly, and the agent should be able to notice that on structure alone. This is the trigger everyone forgets and it catches the most calls in practice.
Sentiment triggers. The caller is audibly frustrated. Worth having, worth keeping conservative, because a false positive here just means a human gets a call they did not strictly need, which is a cheap mistake compared to the alternative.
What the handoff has to carry
A transfer that dumps a confused human into a live call with no context is barely better than a dropped call. The person picking up starts by asking the caller to repeat everything, which is exactly the experience the business was paying to avoid.
So every handoff I build carries a payload: who is calling and their callback number, why they called in the agent's own words, what has already been established, and what triggered the escalation. Where the receiving team works out of a CRM, that lands as a note on the contact before the phone rings. This is one of the jobs the dashboard and CRM layer behind the agent exists to do, and it is why I keep insisting the agent stays thin while the systems behind it stay smart.
The callback number matters more than anything else in that payload. Get it early, get it confirmed, and treat it as the one field the agent is genuinely patient about collecting. If the transfer fails, the number is the only thing standing between the business and a lost lead.
Nobody picks up, which is the normal case
Here is the part that gets designed last and breaks first. Transfer is a request, not a guarantee. The line is busy, the team is with a customer, it is 7pm, someone is on holiday. If the agent's plan was "transfer to a human" and no human answers, the caller gets silence or a voicemail beep and the whole system has failed at the one call it was supposed to protect.
What I build instead is a ladder with an ending:
- Try the primary destination, with a short ring window rather than a long one, because a caller listening to ringing is spending patience they arrived with.
- Try the fallback, if the business has one. On-call phone, second location, an owner's mobile after hours.
- If nobody picks up, the agent comes back on the line and says so honestly. Not "please hold" again. Something closer to what a person would say: everyone is with a customer right now, here is when someone will call you back.
- Capture the callback details, confirm the timeframe, and end the call cleanly.
- Fire the escalation into whatever the team actually watches. That is where the automation layer earns its place, and it is the pattern I described in connecting Retell to n8n: the agent does the talking, n8n does the alerting, the SMS, the ticket and the follow-up sequence.
Step three is the one clients push back on and it is the one that saves them. An agent that admits nobody is available and commits to a callback keeps the lead. An agent that keeps trying to transfer loses them.
The other thing to watch here is how the wait feels. Ring time and lookup time are both spent by the caller, so the latency budget applies to the handoff too. Speak before the silence, not after it.
Warm, cold, and when the difference matters
A cold transfer connects the caller to the destination and the agent is gone. A warm transfer means the receiving person gets told what they are picking up before the caller is connected.
Cold is fine for high volume routing where the destination knows what it is for, like a caller who just wants the parts desk. Warm is worth the extra seconds where the receiving person needs to arrive prepared: emergencies, complaints, anything clinical, anything where the caller has already explained themselves once and would be furious to do it twice.
The tiebreaker I use is simple. If making the caller repeat their story would damage the relationship, transfer warm. Otherwise transfer cold and keep the call fast.
Why I treat this as a safety problem, not a UX problem
Most of my thinking about escalation did not come from voice AI. It came from years of building systems where a confident wrong answer was the worst possible output.
In the clinical eye-tracking work I led at Nystag, the headset could produce a number for every session, but the honest system was the one that could say the capture was not good enough and hand the judgement back to a clinician. In the public safety XR work at ARCortex, an overlay showing stale data with full confidence was more dangerous than one that admitted it had lost tracking.
A voice agent is the same class of system. It will be wrong sometimes, and the design question is never how to prevent that entirely. It is what the system does at the edge of its competence. Escalation is that answer, which is why a well built handoff is the strongest signal that an agent was built by someone who has run one in production.
The short version
- Build three exits: transfer live, take a message and escalate, refuse and route.
- Write the exit conditions before the happy path, from the question "which calls would make you turn this off".
- Trigger on content, on explicit request at any point, on repair loops, and conservatively on sentiment.
- Carry context into the handoff, and get the callback number early.
- Plan for nobody picking up, because that is the normal case, and end that path with an honest promise instead of a hold.
- Transfer warm when repeating the story would cost the relationship.
If you are earlier than this and still deciding what the agent should even be, I wrote a separate piece on choosing between a chat agent and a voice agent based on where you are actually losing people.
I build production voice and chat agents on Retell, wired into n8n, GoHighLevel and Twilio, for teams who need the phone answered properly and the hard calls routed to the right person. If that is you, more about my background here, or book a call.