What Does Best-In-Class Voice AI Design Sound Like?
Use the voice quality checklist to build agents that can detect, interpret, and respond to how humans communicate and actually talk.
Key Takeaways
When someone talks to a voice AI agent, they can’t scan a screen, reread an answer, or skip ahead to find what they need. Voice interactions are inherently ephemeral, requiring people to listen actively and hold information at the front of their mind. That mental load can be a burden and break trust if the design of the conversation doesn’t account for how humans communicate.
Unlike the predictable interactive voice response (IVR) systems that we’re all familiar with, voice AI systems generate responses based on each caller’s situation. That requires a more sophisticated understanding of how to design voice agents to grasp what a caller needs, detect their emotional state and context, and be able to respond appropriately.
Voice AI turns the conversation itself into the interface. Let’s explore the factors that contribute to a seamless voice-only experience and the quality checklist you can use when you design your agent.
Here’s what we’ll cover:
What good voice AI interfaces need
Learn from real conversations
Use the voice quality checklist
How to apply voice design in Agentforce
Design for real conversations
What good voice AI interfaces need
With a graphical user interface (GUI), people can move through an experience at their own pace. A voice user interface (VUI) unfolds in real time through spoken conversation and communication patterns. For example, a customer may never say, “I’m stressed,” but it doesn’t mean there aren’t other signals, including: pauses and hesitation, interruptions and repetition, pacing and word choice, and changes in emotional language and tone.
Agent behaviors might also cause friction: A three-second pause by the agent can feel much longer when there’s no indication of what’s happening. A repeated question can make a simple task feel stuck. An unnecessary confirmation can add seconds to an interaction that already feels long.
A well-designed voice agent should make it easy for customers to resolve their needs. That might mean recognizing when information has already been provided, recovering quickly when something goes wrong, or keeping the conversation moving while the agent processes information. Giving people enough information to understand what the system is doing creates voice AI trust.
Learn from real conversations
Analyzing and evaluating call recordings and annotated transcripts between human agents and callers are essential to understanding how people naturally communicate in a given context. People bring their own expectations, habits, and emotions to every interaction, which can affect how a given person responds to an AI agent. A question that sounds natural in theory may introduce complexity in practice. If the question is straightforward, keep the language clear and straightforward: “Was there an injury, yes or no?”
Customers typically report symptoms, not root causes. They say the agent was “robotic,” “slow,” or “frustrating.” They might mention that it kept asking the same question or that they weren’t sure whether it was still listening. Those observations are valuable, especially because technical success doesn’t always equal user success.
Consider these insurance intake scenarios:
Scenario 1
| Before: Rigid slot filling | After: Designed for real answers |
|---|---|
| AI Agent: “Is this loss related to you, the caller, or a third-party claimant?” Caller: “Both.” AI Agent: “Please say caller or third party.” Caller: “Both.” AI Agent: “Please say caller or third party.” Caller: “Caller.” (gives up, just to escape the loop) | AI Agent: “Is this loss related to you, the caller, or a third-party claimant?” Caller: “Both.” AI Agent: “Got it. This affects both you and a third party. I’ll include both on the claim. Let’s start with your details first.” |
| The caller gives an answer the system wasn’t designed to expect and has to adapt their answer to the system. The agent can’t handle an answer outside its exact-match options, so the claim may end up being filed with inaccurate data. | This gives the agent a way to interpret an unexpected answer, confirm what it understood, and keep the task moving. |
Scenario 2
| Before: Lacks empathy | After: Improved empathy |
|---|---|
| Caller: “There was an accident on site. My employee didn’t make it.” AI agent: “I’m sorry to hear that. Do you have your policy number?” | Caller: “There was an accident on site. My employee didn’t make it.” AI agent: “I’m so sorry – that’s a lot to be dealing with right now. I don’t want to rush you through this. I can connect you directly with one of our representatives right now, I can start the claim with you here, or if you’d rather take more time, I can have someone call you back later. Whatever’s easier for you — what would you like to do?” |
| The one-line acknowledgement lacks empathy. | The AI acknowledges first and then offers several options. |
Reviewing these interactions can expose these moments and show where the voice AI design needs to become more flexible.
Use the voice quality checklist
Based on deep experience in conversation analysis – the study of the systematics of turn-by-turn talk – and IVR design, we’ve developed the Voice Quality Framework, with three tiers of business failures and rules for voice AI design.
- Tier 1 is your foundational layer and table stakes. Ask whether the agent is safe and accurate, can users be understood, can the agent be understood? If your agent is failing here, don’t launch it.
- Tier 2 is where the agent is functional, but there are more invisible churn risks. Do users trust their tasks can be completed efficiently or do they get confused or lost if they deviate from the happy path, and escalate to a human?
- Tier 3 is about the customer experience. Is the agent easy to use? Is it natural and consistent? Does it adapt? This is what separates an agent that works from one people actually want to keep using.
Within the three tiers of failures, there are 15 heuristics – a heuristic is essentially a rule of thumb, or a design principle – grounded in research and real-world experience that tells you: does this experience actually work well for the person using it?
Think of it like a quality checklist. Voice heuristics give us a common language to evaluate agent experiences beyond “Is it good?” and “Is it working?” to “Does this actually feel right to the person on the other end of the line?” The answer might involve changing the pace of the conversation, simplifying a question, providing additional guidance, or changing the path through the task.
| Heuristic | Diagnostic Questions |
|---|---|
| Truthfulness | Does the agent avoid hallucinations, contradictions, and factual errors? |
| Trust | Does the agent stick to what it’s actually allowed to do, protect private info (like account numbers or SSNs), and admit honestly what it can’t help with? |
| Recovery-Awareness | When the conversation goes off the rails, does the agent notice and change tactics – instead of repeating itself or trapping the user? |
| Intelligibility | Does the agent sound clear with good audio quality, a natural pace, and correct pronunciation? |
| Effectiveness | Does the agent actually do what the user asked – including requests to navigate (like going back or starting over) and requests to exit or speak to a person? |
| Responsiveness | Does the agent avoid awkward dead air, letting the user know it’s still working on something? |
| Confirmation Behavior | Does the agent double-check the right amount, confirming risky actions clearly, without over-confirming every little thing? |
| Memory and Context | Does the agent remember what the user already said (or what the company already has on file), instead of asking again? |
| Adaptability | When the user corrects the agent or changes their mind, does it adjust smoothly – without arguing, getting defensive, or repeating the mistake? |
| Decisiveness | Does the agent move the user forward with clear answers and next steps, instead of hedging or leaving the user unsure what to do? |
| Interruption Tolerance | If the user talks over the agent, does it stop and listen gracefully, then pick back up cleanly? |
| Conversation Flow | Does the agent talk naturally, without rambling – and when something goes wrong, does it explain it plainly instead of faking sympathy or laying blame? |
| Consistency | Does the agent keep using the same words, tone, and personality throughout the call, instead of sounding like a different agent each turn? Or like a generic agent using the homogenous and cliched “LLM voice” with no brand distinctiveness or persona? |
| Approachability | Can the agent understand everyday speech (slang, accents, casual phrasing), avoid confusing jargon, and adjust if the user switches languages? |
| Thoroughness | Does the agent finish the job on the call itself, confirm it actually worked, and offer relevant help before saying goodbye? |
The Voice Quality Framework bridges design, product, and engineering, enabling agent evaluation at scale. Instead of relying entirely on manual reviews, teams can use an automated evaluation workflow to analyze calls, score interactions against the heuristics, connect failures to specific turns in a conversation, and surface patterns across sessions.
How to apply voice design in Agentforce
There are three layers in Agentforce where design lives. For behaviors like recovery-awareness and responsiveness, write clear turn rules in Agent Script (“acknowledge within one turn if a task runs long”) instead of vague tone guidance like “be reassuring.” Clear rules lead to consistent behavior; vague ones don’t.
| Layer | Purpose |
| 1. Prose prompt instructions (|) | Controls conversational behavior: e.g., turn shape, end focus, repair phrasing. Written as concrete turn rules. |
| 2. Deterministic logic & variables | Guarantees sequence: e.g., slot-filling variables, retry counters, mandatory summary confirmations, automated escalation logic. |
| 3. Voice channel configuration | Handles sound and timing: e.g., TTS voice selection, custom pronunciations, filler turns (messaging and audio), acoustic barge-in, telephony routing. |
Conversation flow and approachability come down to handling how people actually talk: someone answers three questions at once, or skips straight to what they want. Instead of relying on the agent to remember details from earlier in the conversation, the system tracks them directly, like which booking details are filled in or how many times it’s tried to fix a problem, so nothing gets lost when an answer comes in early or out of order.
That same tracking layer also handles things that must happen every time, like escalating after three failed repairs or never skipping a confirmation. These aren’t left to the agent’s judgment in the moment; they’re set rules the system always follows. This hybrid deterministic-probabilistic approach in Agentforce combines an LLM’s linguistic fluency with logic that guarantees sequence and reliability; no “prompt and prey” here.
Sound and timing are their own layer: which voice speaks, how it pronounces things, and the thresholds behind filler audio and barge-in/speaker overlap. Treat this as its own pre-launch check. Make sure the voice fits the agent’s persona and the timing is tuned, not left at default.
Design for real conversations
Voice AI succeeds when it understands how people actually talk. Real conversations come with hesitation, emotion, interruptions, repetition, and unexpected answers. People don’t bring perfect scripts to a phone call, so the AI agent needs to be ready for the conversation that actually happens. Explicit turn rules, clear recovery paths, and structured heuristics give agents a foundation for adapting in the moment and keeping the conversation moving.
The best voice agents will make callers feel heard, oriented, and in control. When conversation becomes the interface, thoughtful design is what makes that possible. Grounding an agent’s behavior in real human communication patterns helps turn capable technology into something people can actually use with confidence.
