TL;DR: Almost every voice agent dashboard I inherit leads with calls handled and minutes used. Neither number tells anyone whether the agent is working. The five that do: pickup rate, resolution rate, booking rate against callers who wanted to book, transfer success rate, and cost per booked appointment. Define the denominator for each one before the first live call, because a metric two people define differently is worse than no metric at all. Then measure latency at the worst turn rather than the average, and read the failures instead of a random sample.
A client called me a month after launch to say the agent was doing great. He had the number in front of him: 412 calls handled. I asked how many appointments came out of it. He did not know, and the dashboard could not tell him, because it had been built to show what the agent did rather than what the business got.
That gap is the normal state of voice AI reporting. Platform analytics are built around the platform's unit of work, which is a call and a minute of audio. The business's unit of work is a booked job. Nobody wires the two together by default, so the agent runs for weeks in a state where nobody can honestly say if it is earning its keep, and the first real question from the owner ends the project.
Activity metrics are not outcome metrics
Calls handled, total minutes, average call duration and message counts are activity. They tell you the system is running. They do not tell you it is working, and some of them move in the wrong direction for good reasons. An agent that gets better at resolving calls quickly reduces average duration and total minutes. Read as a performance metric, competence looks like decline.
The distinction I hold to is simple: if a number can go up while the business gets nothing, it is not a performance metric. Put it in a health panel where it belongs, and keep the top of the dashboard for outcomes.
Define the denominator before you define the metric
This is the part that gets skipped, and it is the part that decides whether anyone trusts the dashboard six weeks in. Every rate needs an agreed answer to a boring question: which calls count?
A raw call log is full of things that are not calls. Silent connects that drop in two seconds. Wrong numbers. Robocalls and voicemail-drop spam. The client's own staff testing the agent. The same customer calling three times in ten minutes because the first attempt was cut off. Leave all of that in the denominator and every rate on the dashboard is quietly deflated. Strip it out with no written rule and the numbers become unfalsifiable, which is worse.
I learned this the hard way on analytics work rather than voice work. Building the admin dashboard for Amazon sellers at Bettershop, the hardest problem was never the query layer, it was that profit meant four different things to four people. The fix is the same here: write the definitions down, apply them in one place in the pipeline, and show the exclusion count on the dashboard so the filtering is visible instead of magic.
The five numbers I put in front of a client
Pickup rate. Of inbound calls that reached the number, how many did the agent actually answer and start speaking on. This is infrastructure health, not AI performance, and it catches the failures that matter most: a broken forwarding rule, a provider outage, a misconfigured number. If this is not at or near the top, nothing below it means anything. Most of what threatens it lives in the telephony layer.
Resolution rate. Of qualifying calls, how many ended with the caller's reason for calling actually handled, with no human needed afterwards. This is the closest thing to a single quality score, and it is the number I care about most in the first month. It is also the one that requires real outcome tagging rather than a call status, which I get to below.
Booking rate, against intent. Not bookings divided by all calls. Bookings divided by calls where the caller wanted to book. Mixing those two together produces a number that mostly measures your call mix, so it drifts every time marketing changes and tells you nothing about the agent. Split by appointment type once you have volume, because a low rate is usually concentrated in one type with a scheduling constraint the agent handles badly.
Transfer rate and transfer success. These are two different things and conflating them hides the ugliest failure in voice AI. Transfer rate is how often the agent decided to escalate, which is a design metric: too high means the agent's scope is too narrow, too low often means it is bulldozing past callers who asked for a person. Transfer success is whether a human actually picked up. An agent that hands off correctly into an unanswered line has not helped anyone, and the caller experience is worse than voicemail. I build the fallback ladder for that case deliberately, as described in the human handoff piece, and then I measure whether the ladder is being used, because heavy use of the last rung is a staffing problem the client needs to see.
Cost per booked appointment. Total usage cost divided by appointments booked. This is the only number most owners genuinely care about, and it is the one that survives contact with a skeptical accountant. It also makes the platform choice concrete: a cheaper per-minute vendor that resolves fewer calls is not cheaper. The per-minute components behind it are broken down in what an AI receptionist actually costs.
Outcome tagging: the agent labels its own calls, and you verify it
None of the above works without knowing what happened on each call, and the platform does not know. It knows the call ended.
What I do is structured post-call extraction: after every call, a small extraction step reads the transcript and emits a fixed schema. Outcome from a closed list, intent, whether a booking was attempted and whether it succeeded, whether an escalation happened, and any unresolved question. That object goes into the CRM against the contact and into the dashboard row, through the same n8n pipeline that handles the rest of the post-call work. Fixed vocabulary matters more than richness here. Free-text summaries are pleasant to read and impossible to aggregate.
Then the important part: the agent's self-report is evidence, not truth. A model summarizing its own conversation is biased toward reporting success. So a sample of calls gets labelled by hand each week and compared against the extraction, and the disagreement rate between the two is itself a metric. When the agent says resolved and a human reading the transcript says the caller gave up, that is the most valuable data in the whole system.
That instinct comes straight out of clinical work. Building eye-tracking diagnostics on the Vive Focus 3, the discipline that mattered was never trusting a derived number without a path back to the raw signal. Same rule here. Keep the transcript and the recording linked to every metric row, so any number on the dashboard can be opened and checked.
Measure latency at the worst turn, not the average
Response time belongs in the health panel, but only if it is measured the way callers experience it. Average turn latency is a comfortable number that hides the problem. Callers do not remember the average, they remember the one turn where nothing happened for four seconds and they said hello twice.
So I track the tail, the slowest turns in a call, and I track dead air events over a threshold as a count rather than an average. Those cluster around exactly the moments I expect: availability lookups, CRM writes and any tool call that was allowed to block the conversation. Watching the tail is what tells you which tool to move off the critical path, which is the whole argument of the latency budget piece.
Read the failures, not a random sample
Every serious voice deployment needs a human reading transcripts every week. The mistake is reading random ones, which mostly means reading successful calls and feeling good.
The queue I actually work through: calls that ended abnormally, calls where the agent repeated itself or asked the same question twice, calls under twenty seconds that were not spam, every escalation, and every booking that failed after being attempted. That list is where the next prompt change comes from. A repair loop in a transcript is a node with a weak exit condition, and it is fixed structurally rather than by adding another sentence of instruction, for the reasons in sizing conversation flow nodes.
Where the numbers should live
Metrics the client cannot see do not build trust, they build dependency on me. That was the argument behind the client dashboard I built at Tested Media alongside the CRM, described in the back half of a voice agent build, and it is the same principle behind white-labelled portals in VoiceDash. The owner should be able to open one page, see what the agent did last week in outcome terms, and click into any call that looks wrong.
Six numbers, one filter definition, and a transcript link on every row beats a beautiful analytics suite nobody can interpret.
The short version
- Activity metrics belong in a health panel. Outcomes go at the top.
- Write down which calls count before launch, and show the exclusions.
- Track pickup, resolution, booking against intent, transfer rate plus transfer success, and cost per booked appointment.
- Extract a fixed outcome schema after every call, and treat the agent's self-report as a claim to be audited.
- Sample-label by hand weekly and track the disagreement rate.
- Watch latency at the tail, and count dead air events instead of averaging them.
- Read failed calls, not random ones.
- Give the client the numbers directly, with a way to open any call behind them.
An agent nobody can evaluate gets switched off during the first budget review, no matter how well it talks. An agent with an honest number next to it gets a second phone line.
I build production voice and chat agents on Retell, wired into n8n, GoHighLevel and Twilio, with the CRM, dashboards and reporting layer behind them actually working. More about my background here, or book a call.