NNabeel Hassan

Blog · September 18, 2026 · 9 min read

When Everyone Calls at Once: What Actually Breaks in a Voice Agent Under Load

By Nabeel HassanAI Engineer · ICPC World Finalist

TL;DR: "An AI agent answers every call at once" is true and it is also the least interesting part of the system. The agent almost never runs out of capacity. The back half does. Under a spike, the CRM write slows down, the calendar lookup queues behind nine other calendar lookups, and what the caller hears is not a busy tone, it is four seconds of silence in the middle of a sentence. The work is capacity planning for the tool layer, timeouts with a fallback the caller can live with, idempotent writes so retries do not double book, pacing on anything outbound, and a stated overflow path for the ceiling you will eventually hit.

Every voice agent pitch includes the same line, usually mine: ten simultaneous callers are ten answered calls. It is the one claim that no human front desk can match, and it is the reason an AI receptionist beats an answering service on the days that actually matter.

Then you ship, the client sends one campaign, and you learn that concurrency is not a property of the agent. It is a property of the weakest thing the agent depends on.

At Fortell AI I built production agents for hospitals, vet clinics, estate agents and car garages in the UK, which is four businesses with four completely different spike shapes. At Tested Media I work on voice and chat agents for the CallSetter AI product with GoHighLevel behind it, where campaigns deliberately manufacture the spike. This is what actually breaks, in the order it breaks.

Four ceilings, and only one of them is the agent

The platform concurrency cap. Every voice platform has a maximum number of simultaneous calls on your plan, usually generous enough that a single small business never touches it. Worth checking once and then forgetting, unless you run many clients under one account, where it stops being a footnote: one client's campaign can starve another.

Telephony. The number, the trunk, and whatever sits between the public network and the platform. A limit here shows up as callers hearing a busy tone rather than anything you built, which makes it the most confusing failure to debug: your dashboard looks healthy and your logs show nothing, because the call never arrived.

Model and speech provider limits. Rate limits on the language model and the voice, invisible under normal traffic and arriving under a burst as a slow or failed turn mid conversation. On a managed platform this is mostly someone else's problem, right up until a regional incident makes it yours.

The back half. The CRM, the calendar, the knowledge lookup, the automation instance. This is the real ceiling, it is far lower than the other three, and it is entirely inside your build. Nobody hits a platform cap on a Monday morning. Everybody hits a calendar API that was happy at one request every two minutes and is now getting nine at once.

What a spike actually looks like

The word "spike" makes people think of a doubling of daily volume. That almost never happens. What happens is that the same daily volume arrives inside a ten minute window.

The shapes I have built for are all real and all different. A vet clinic gets Monday at nine, when the weekend's worries all dial at once. A car garage gets the first hard frost of the year. A hospital switchboard gets whatever was just announced. An estate agent gets the ninety minutes after a listing goes live. A campaign send, the only one you control, drops a flood of callbacks into a window you chose.

Rough arithmetic is enough to plan with. Concurrent calls sit around calls per hour multiplied by average call length in minutes, divided by sixty. Sixty calls an hour at four minutes each is about four concurrent, which sounds trivial, but arrivals cluster, so the honest planning number is roughly three times that inside the busiest ten minutes. Twelve simultaneous calls, each firing two or three tool calls, is thirty odd requests landing on a CRM in a few seconds. That is the number that matters, and it is not the number anyone writes down.

Load does not break the agent, it slows the back half

Here is the failure mode nobody expects. The agent does not refuse the call or crash. It waits.

A tool call that takes 600ms in isolation takes four seconds when eleven other calls want the same thing, and the caller experiences that as the agent going silent mid sentence. Everything I have written about the latency budget of a voice call applies with a multiplier under load, because every queued request spends its wait inside somebody's silence.

So the rule I build to now: every tool call gets a timeout well below the caller's patience, and every timeout gets a path that is not an apology.

Three seconds is my usual ceiling, and the fallback has to be a sentence the agent can say without sounding broken. This is the part of function and tool calling that only reveals itself in production, because in testing your tools are always fast.

Three bugs that only exist when calls overlap

The double booking race. Two callers are offered the same slot in the same four seconds, both accept, both writes succeed. Nothing is technically broken and the clinic has a problem at ten past three. Fixes in order of preference: let the calendar enforce the conflict and handle the failure gracefully rather than checking first and writing later, hold the slot when it is offered, and offer fewer slots so fewer callers collide.

Retries that duplicate. Under load, requests fail and get retried, and a retry without an idempotency key is a second appointment, a second lead, a second confirmation text to a person who is now annoyed. Every write in the post call automation layer needs a stable key derived from the call, not a fresh one per attempt. This bug is invisible at one call at a time and obvious at ten.

Shared state in the automation. A workflow that stores "the current caller" anywhere global will serve caller A's details into caller B's confirmation when the two overlap. It survives to production because single threaded testing never exposes it, and the consequence is a privacy incident rather than an error in a log.

Building a back half that survives a burst

The highest value change is the simplest: the webhook receiving the call event returns 200 immediately and does the work afterwards. If the n8n workflow behind the agent does eleven sequential things before it answers, a burst leaves you holding open connections while one slow CRM builds a backlog that outlives the spike.

Three more that earn their keep:

Retry with backoff, not with enthusiasm. A 429 answered by an instant retry is how you turn a brief rate limit into a sustained one.

Cache the layer that does not change. Hours, prices, services and policies belong in the static tiers of the knowledge base, not in a live lookup. Every question you can answer without a network call is one less request in the queue at nine on Monday.

Size the automation instance for the peak. A small self hosted n8n box is fine at four concurrent executions and not at forty. A five minute decision that people discover three weeks late.

Outbound is the spike you create yourself

Inbound spikes happen to you. Outbound spikes are your own doing, which makes them both easier to prevent and easier to cause at scale. The dial queue in front of an outbound agent needs a pacing limit and a concurrency cap per campaign, always, even when the platform would happily go faster.

Two reasons. Every outbound call consumes the same downstream capacity your inbound callers need, so a campaign that saturates the CRM degrades the calls worth the most. And a burst of calls from one number is exactly the pattern carriers flag, which turns into the spam labelling problem I covered in the phone number setup guide. Slow pacing is not a limitation, it is protection for the number.

What happens when you do hit the ceiling

Something has to. The design question is whether the overflow behaviour is chosen or accidental.

Accidental looks like a busy tone, or a silent drop into a voicemail box nobody has checked in years. Chosen looks like an overflow rule: past the cap, calls forward to a human, to an out of hours service, or to a short capture flow that takes a name and a number and sets up a callback. The human handoff path you already built for the agent's hard cases is the same path, pointed at a different trigger, and it belongs in the handover pack because the day it fires is the day the client is least able to work out what happened.

Testing it without burning minutes

Real load tests on a phone system are awkward and cost real money per minute, so I do not test the whole call path at scale. I split it.

The back half gets tested properly. Replay a captured call payload thirty times concurrently against the automation and watch the CRM writes, the calendar, the retries and the duplicates. That is where every bug in this article lives, and it costs nothing but a script.

The call path gets tested for overlap, not volume. Four simultaneous calls, deliberately awkward, is enough to catch shared state and slow tool calls. Then it joins the scenario set I re-run before any change goes live, because concurrency bugs come back quietly.

Under a real spike, three numbers tell you the story: peak simultaneous calls, the p95 latency of your slowest tool call, and the failure rate of the post call webhook. The rest of the metrics that matter can wait until the week is over.

The short version

The agent is not the bottleneck. The CRM, the calendar and the automation instance are, and they fail by getting slow rather than by going down, which the caller hears as silence.

Size the tool layer for the busiest ten minutes rather than the daily average, put a timeout and a human sounding fallback on every tool call, make every write idempotent, keep per call state out of anything global, pace anything outbound, and decide in advance what happens to call number thirteen.

Ten simultaneous callers are ten answered calls, but only if you built the ten answers.


I build production AI voice agents and the automation, CRM and telephony layer behind them for founders across the US, UK and Europe. More about my work here, or book a call.

FAQ

How many calls can an AI voice agent handle at the same time?

The voice platform's own concurrency cap is rarely the limit, and for a single small business it is usually never reached. The real ceiling is the back half: the CRM, the calendar, the knowledge lookup and the automation instance behind the agent. Twelve simultaneous calls firing two or three tool calls each puts roughly thirty requests onto a CRM within a few seconds, which is the number worth planning against. Size that layer for the busiest ten minutes rather than the daily average, since concurrent calls run around calls per hour times average call minutes divided by sixty, and arrivals cluster at roughly three times that inside a peak window.

What breaks first when a voice agent gets a spike in calls?

Latency, not availability. A tool call that returns in 600ms in isolation can take four seconds when a dozen calls compete for the same CRM or calendar API, and the caller experiences that as the agent going silent mid sentence. The fix is a timeout on every tool call well below the caller's patience, usually around three seconds, with a fallback the agent can say without sounding broken: take the details and confirm by text, answer from the cached knowledge layer, or treat the caller as new and reconcile afterwards.

Can two callers book the same appointment slot with an AI agent?

Yes, if the booking flow checks availability and then writes, because two overlapping calls can pass the check in the same few seconds. Let the calendar system enforce the conflict and handle the failure gracefully instead, hold the slot at the moment it is offered, and offer fewer slots so fewer callers collide. Two related concurrency bugs matter just as much: retries without an idempotency key create duplicate appointments and leads, and any per call state stored globally in an automation workflow can serve one caller's details into another caller's confirmation.

Building something in this space?

I take on AI-agent, automation and product work directly — scoped fast, shipped fast.

Book a discovery call →

Keep reading