TL;DR: Hiring an AI voice agent developer goes wrong when you judge the demo call and skip everything around it. A five-minute demo on a quiet line proves almost nothing. What separates a production voice agent developer from a prompt tinkerer is what happens off the happy path: transfers that fail, callers who interrupt, details that have to land correctly in your CRM, calendar bookings that must not double up, and a way to test changes before they reach real callers. Ask to call something they built that is live, ask how they test it, ask what happens after the call ends, and give a small paid trial on your real use case. I build these for a living, production voice and chat agents for hospitals, real estate, vet clinics and car garages at Fortell AI and the CallSetter AI agents at Tested Media, so this is the checklist I would want a client to use on me.
Most people hiring for this are not voice AI people. They run a clinic, a real estate team, an agency or a startup, and they have heard that an AI agent can answer the phone. That is fine. It just means the hiring process tends to over-weight the one thing that is easy to judge: a demo call that sounds good.
Every platform can make a demo sound good. Retell, Vapi and the rest have made the first ten minutes of a voice agent almost trivial. The hard part is the next few months, when the agent meets real callers on real phone lines and has to be right about things your business depends on.
Start with what the agent actually has to do
"Voice agent developer" covers a few different jobs. Before you post anything, decide which one you need.
- Inbound receptionist. Answers calls, handles common questions, books or captures details, transfers when needed. This is most clinics, garages and service businesses. You need someone careful about capturing caller details, booking into a calendar and handing off to a human.
- Outbound or lead follow-up. Calls new leads, reactivates old ones, confirms appointments. This is a different discipline with its own consent rules, retry cadence and answering machine handling. I wrote up how outbound differs from inbound.
- A voice product you sell. You are an agency or a SaaS putting agents in front of many clients. Now you also need multi-tenant thinking, client dashboards and a way to change one client's agent without breaking another's. That is closer to what I built for CallSetter AI at Tested Media and what VoiceDash exists for.
A developer who is excellent at one of these is not automatically right for another. A good one will tell you which they have actually shipped.
What to look for in their past work
Ask to call something live
The most useful request is simple: "Can I call an agent you built that real callers use?" Not a demo number set up for sales. A live one, or a recording of real calls with the client's permission and personal details removed.
Live agents carry the scars of real traffic. They handle people who talk over them, callers in cars, accents the developer did not plan for and questions that are not in the script. If everything they can show you is a demo, you are hiring someone who has not yet learned what real callers do.
Ask what broke after launch
Everyone who has shipped a voice agent has an answer to this. Mine include transfers that rang out because nobody was at the desk, callers reading phone numbers in a rhythm the agent split wrongly, and a small wording change that quietly changed how the agent handled a different branch of the conversation. That last one is why I now treat regression testing a voice agent as part of the job, not an extra.
If a candidate says nothing broke, either they did not ship much or they were not the one reading the call logs.
Ask them to walk the whole system, not the prompt
A production voice agent is not a prompt. It is a phone number, a speech pipeline, a model, a set of tools it can call, a CRM or calendar it writes into, automation that runs after the call, and someone watching the results. Ask the candidate to draw that for one of their projects.
You do not need to follow every detail. You are listening for whether they think in systems. At Tested Media the system was Retell voice agents, a chat-dash client dashboard and GoHighLevel as the CRM for campaigns and lead management. The agent was maybe a third of the work. A developer who only talks about the prompt has built a third of a product.
The technical questions that separate production from prototype
These are the questions I would ask, and what a strong answer sounds like.
"How do you test a change before it goes live?"
A weak answer is "I call it a few times." A strong answer has a set of saved scenarios pulled from real calls, a mix of automated checks and listening on a real phone, and a rule about who publishes changes and how to roll back. At Fortell AI I built a pipeline that used Claude Code and Comet browser automation to build and test agents in minutes, and I wrote up how that pipeline works. You do not need that level of automation for one small agent, but you need someone who knows why testing matters when the thing under test is a conversation.
"What happens after the call ends?"
This is where most of the business value is, and most of the silent failures. The call summary goes somewhere, a contact is created or updated, a booking is confirmed, a text is sent, a staff member is notified. A strong candidate talks about what happens when one of those steps fails: retries, duplicate protection, alerts. I have a whole post on post-call automation failures because this is where agents that sound perfect still lose leads.
"How does the agent call your systems during the conversation?"
Checking availability, looking up a booking, creating a lead. These are function calls, and they need timeouts, sensible fallbacks and something the agent says while it waits. Ask what the agent does when the calendar API is slow or down. "It just works" is not an answer.
"How do you keep it fast?"
Callers notice a pause of more than about a second. A strong candidate can explain where the time goes, speech recognition, the model, tool calls, text to speech, and what they trade off to stay inside a latency budget. They should also have an opinion on which model to use and why it is not always the biggest one.
"What should the agent refuse to do?"
Good voice developers spend real time on the edges: callers trying to get the agent to say something it should not, spam and sales calls, upset callers, and questions the agent is not qualified to answer. For a hospital or clinic this matters a lot. When I built agents for hospitals at Fortell, the most important part of the design was what the agent would not do and when it would hand off to a person.
Red flags I would watch for
- Only demo numbers, no live agents. They have not met real callers yet.
- "The prompt handles that." For bookings, transfers or data capture, the answer should involve tools, validation and checks, not instructions in prose.
- No plan for compliance. Recording consent, AI disclosure and, for texting in the US, A2P 10DLC registration. A candidate who has never heard of call recording consent or A2P 10DLC will discover them on your time.
- Everything locked in their own accounts. The phone number, the platform workspace and the automation should belong to you. I wrote about what you should own when someone builds your software, and voice is no exception.
- A fixed price before they understand your calls. Nobody can quote a voice agent accurately without knowing your call types, volumes and systems. A good developer runs a discovery call first.
- No mention of what happens in week two. The first two weeks of real calls always reveal things. A candidate who plans to hand over on launch day and disappear has not done this before. My notes on the first two weeks live explain why.
How to run a trial that tells you something
Unpaid test projects filter for free time, not skill. Pay for a small, real piece of work instead. Good options:
- Build an after-hours agent for one call type, with one real tool, such as capturing a lead into your CRM.
- Take your existing agent and fix one known problem, with a before-and-after set of test calls.
- Write a short scoping document for your full use case, including what they would not automate.
Judge the communication as much as the result. Did they ask about your actual callers? Did they push back on anything? Did they show you how they tested it? Did you end up with access to everything they built? The written handover tells you a lot about what working with them for six months will feel like, which is also why I keep a handover checklist for every client agent.
Freelancer, agency or platform?
There are three broad routes. A freelancer gives you direct access to the person building it and is usually the best fit for one business with a few call types. An agency makes sense when you need many agents, ongoing support across clients or a team covering several time zones. A no-code platform with templates can be enough for a very simple use case, as long as you accept its limits. I compared the trade-offs in more depth in freelancer vs agency vs in-house.
Whichever you pick, the questions above still apply. A template still has to book correctly, transfer correctly and write the right data into your systems.
The short version
Do not hire on the demo call. Ask to call something live, ask what broke, and ask them to walk the whole system from phone number to CRM. Probe testing, post-call automation, tool calls, latency and refusals. Watch for prompt-only answers, missing compliance and accounts you will not own. Then pay for a small trial on your real use case and judge the handover as much as the agent.
I build production AI voice and chat agents on Retell AI, wired into n8n, GoHighLevel and Twilio, for businesses and agencies in the US, UK and Europe. If you want someone to build yours, or to review an agent you already have, more about my background is here, or book a call.