Why did my AI voice agent book an appointment that never appeared in the calendar?

Almost always because the automation behind the call failed after the agent had already promised the booking out loud, and nothing surfaced the failure. The three usual causes are that the call-ended event never reached the automation at all (a webhook that was down, redeployed or rate limited), that the workflow ran partway and died between two writes, or that a credential quietly expired so the calendar write failed while every other step succeeded. None of these produce a bad-sounding call, which is why they go unnoticed for days. The fix is structural rather than a better prompt: store the raw event before processing it, order the writes so the booking happens first, and turn a failed booking into a visible task for a human rather than a log line nobody reads.

How do you stop a voice agent webhook from creating duplicate bookings or contacts?

By making every write keyed on something stable rather than a blind create, so a re-delivered or replayed event lands on the same record instead of a second one. I key the call record on the platform's call ID, the contact on the normalized phone number rather than the name (names get transcribed several different ways for the same caller), and the booking on a deterministic combination of contact, appointment type and start time that is checked immediately before writing. Side effects that are not naturally idempotent, like a confirmation text, get an explicit marker on the record saying they have been sent, checked inside the same workflow that sends them.

How do you monitor an AI voice agent's automations in production?

Error alerting on its own is not enough, because the worst outage does not throw anything: the workflow was deactivated during an edit, the webhook URL changed, or delivery stopped, and the automation is not failing so much as it is simply not running. So alongside classifying failures into transient (retry with backoff and a hard attempt cap) and permanent (never retry, escalate immediately), I run a daily reconciliation job that pulls the list of calls the voice platform says happened and compares it against the records the automation actually created, then alerts on the difference. Keeping the raw event payloads makes the recovery cheap, because a day of missed events can be replayed safely.

The Call Went Perfectly and Nothing Happened: Post-Call Automation That Fails Loudly

TL;DR: The scariest failure in a production voice agent is not a bad call. It is a perfect call where nothing happened afterwards. The caller was booked, thanked and hung up, and the calendar event, the CRM record and the confirmation text never existed. The call layer has a human watching it in real time; the post-call layer has nobody. So it has to be built to fail loudly: accept the event durably before processing it, key every write on something stable so a replay cannot double-book, order steps by what hurts most to lose, turn partial failures into a human task instead of a log line, retry only what is safe to retry, and alert on absence rather than errors. Here is how I build that layer.

Every voice agent I ship has two halves. The half people demo is the conversation. The half that decides whether the client renews is what happens in the ninety seconds after the caller hangs up: the booking gets written, the CRM contact gets created or updated, the confirmation text goes out, the transcript and outcome land somewhere the client can read them.

That second half runs unattended, on other people's APIs, with no one listening. When it breaks, it does not look like a failure. The recording sounds great. The transcript reads well. The agent said "you're all set for Tuesday at two." And Tuesday at two, nobody is expecting them.

I have chased this exact class of bug across vet practices, clinics, garages and lead-gen campaigns, and it is almost never the model. It is the plumbing behind the model, which is n8n in my stack. These are the rules I now build that plumbing with.

Why the post-call layer fails differently

During the call, failure has a witness. If a mid-call availability lookup hangs, the caller hears silence and complains, and that is a bug report arriving in real time. That is why I spend so much attention on the latency budget for anything that runs while somebody is on the line.

After the call, nobody is watching. The caller assumes they are booked. The business assumes the system works because it worked in testing. The gap between those two assumptions can run for a week before anyone notices, and what surfaces first is usually a no-show or an angry phone call, not an alert.

The failures come in three shapes, and they need three different defences:

The event never arrives. Your webhook was down, redeployed, rate limited, or the platform's delivery failed. Nothing ran at all.
The workflow half-ran. The calendar write succeeded, the CRM write threw, and the workflow died holding a booking that only exists in one place.
The workflow ran twice. A retried delivery created a duplicate contact, a second calendar event, and a second confirmation text to a caller who now thinks they have two appointments.

Rule 1: accepting the event and processing it are two different jobs

The most common mistake I see in webhook-driven automations is a single workflow that receives the call-ended event and immediately starts doing real work: querying a calendar, writing to a CRM, sending SMS, and only then responding. Every extra second in that chain is a second in which a timeout or a redeploy loses the event permanently, because the payload existed nowhere except in flight.

Split it. The receiving workflow does two things: write the raw payload somewhere durable keyed on the call ID, and respond. Everything else reads from that stored record. It costs one extra hop and it converts an entire class of catastrophic loss into a delay, because once the payload is on disk the work can be re-run whenever you like.

This also gives you something you cannot buy later: a raw event history. When a client asks in week three why a specific caller was never contacted, the stored payload is the only artifact that can answer honestly. Retell's own dashboard tells you what the call did. It does not tell you what your automation did with it.

Rule 2: every write needs a key that survives a replay

Once events can be re-delivered or replayed, correctness stops depending on how many times a workflow runs. That means no write is allowed to be a blind create.

In practice I key three things separately:

The call record on the platform's call_id. That one is easy and it is the one people usually get right.
The contact on the normalized phone number, never the name. Names get transcribed several different ways for the same person, which is exactly why identifiers get captured wrong on a phone line. Phone number is the only thing the call is guaranteed to know.
The booking on a deterministic combination of contact, appointment type and start time, checked immediately before writing.

The third one matters most because it is the write with real world consequences and it is a race, not a lookup. Two deliveries of the same event, or a caller who rings twice in a minute, both end at the same place: the write must be the thing that decides, and it must be able to recognise its own previous result.

The same logic applies to outbound side effects. A confirmation text is not idempotent by nature, so guard it explicitly with a marker on the record saying it has been sent, and check that marker inside the same workflow that sends it.

Rule 3: order the steps by what hurts most to lose

A workflow that dies halfway through does not choose where to stop. You choose, by ordering.

I put steps in descending order of how expensive they are to lose:

The booking, because the caller has already been promised it out loud.
The CRM record, because it is how the business finds the caller again.
The confirmation text.
The dashboard, analytics and internal notifications.

Reversed, that same workflow can fail after texting somebody a confirmation for an appointment that was never written. Ordered this way, a mid-workflow failure loses a dashboard row, and I can rebuild a dashboard row from the stored payload any time.

The corollary is that nothing early in the chain should ever be blocked by something late in it. A failing analytics webhook must not be able to prevent a booking.

Rule 4: a partial failure is a task, not a log line

Here is the case that actually costs money. The agent told the caller they were booked. The calendar write then failed, because a token expired or the calendar API was having a bad afternoon.

Retrying is right for about thirty seconds. After that the honest answer is that a human has to close the loop, because a promise was made on the phone that the system could not keep. So the failure branch does not just log. It creates a visible task in the CRM the client already looks at, with the caller's number, what they asked for and the transcript attached, and it flags the call in the client dashboard rather than letting it sit in a green row.

That principle came from clinical XR work rather than voice: on the eye-tracking diagnostics build at Nystag, the rule was that a measurement the system was not sure about must be surfaced as uncertain, never quietly averaged into a clean-looking result. A call whose booking silently failed is the same defect. The system knows something is wrong, and the interface says everything is fine.

Rule 5: retry only what is worth retrying

Blanket retries are how you turn one problem into two. Before anything retries, classify the failure:

Transient: timeouts, 5xx responses, rate limits. Retry with backoff, a hard attempt cap, and idempotency keys already in place from rule 2.
Permanent: authentication failures, validation errors, a calendar that no longer exists, unregistered SMS traffic. Never retry these, because the fifth attempt fails identically and burns the window in which somebody could have fixed it. Escalate immediately.

The permanent bucket is the one worth wiring alerts to, because most of it is configuration that was fine at launch and drifted: a revoked integration, a client who deleted a calendar, an A2P campaign that lapsed. Those failures are boring, silent, and account for a large share of the "the agent stopped working" messages I get.

Rule 6: alert on absence, not just on errors

Every rule so far assumes something threw. The worst outage does not throw at all. The webhook URL changed, the workflow got deactivated during an edit, the platform stopped delivering, and the automation layer is not failing so much as it is simply not running. Error alerting is silent, because there are no errors.

The fix is a reconciliation job rather than a smarter error handler. Once a day, pull the list of calls the voice platform says happened, compare it against the records your automation created, and alert on the difference.

That is also the cheapest way to catch the subtler version, where the automation runs but produces nothing useful, which is the same discipline behind the metrics I actually track on a live agent. Activity is not outcome. Twenty calls and zero records is a system that is technically healthy and practically dead.

Test the failures before launch, not after

Every one of these rules exists because I have seen the failure. So the pre-launch pass is not only the happy path:

Fire the same call-ended payload twice and confirm you get one contact, one booking, one text.
Revoke the calendar credential and make a real booking call, then check that a human sees a task rather than a green row.
Point the calendar step at a URL that hangs, and confirm the workflow gives up rather than retrying forever.
Deactivate the workflow entirely for an afternoon, then confirm the reconciliation job catches the gap the next morning.
Replay a stored payload from a week ago and confirm it rebuilds cleanly instead of duplicating.

The last one is the real test, because if a day of stored events can be replayed safely, most outages become an inconvenience rather than lost business.

The part clients actually judge

None of this shows up in a demo. What clients experience is whether the thing they were promised on the phone reliably exists afterwards, which is why I treat the client dashboard as a trust instrument rather than a reporting feature. A dashboard that shows a failed booking as failed buys more trust than one that shows everything as fine and is occasionally lying.

The agent is the part everyone looks at. The ninety seconds after the call are the part that decides whether it was worth building.

I build production AI voice agents and the automation behind them for founders across the US, UK and Europe, from Retell agents to the n8n, CRM and telephony layer that has to hold up unattended. More about my work here, or book a call.