Voice agent latency
Latency is the difference between a conversation and a wait. Here is the budget and the breakdown.
- Simba 3.2 produces first audio in under 100ms
- Full pipeline lands under 1 second end-to-end
- Simba 3.2 is #1 on Artificial Analysis
Definition
Voice agent latency is the gap between a caller finishing their sentence and the agent beginning its reply. It is not one number but a sum: speech-to-text, the LLM’s reasoning, text-to-speech first audio, and the network hops between them. Cross about half a second and the caller feels the wait.
The budget
The lower the total, the better. Inside the per-turn budget, each stage takes a share: transcription, reasoning, and speech generation. The art is keeping the total under the threshold where a human perceives the reply as immediate rather than delayed.
The breakdown
Speech-to-text runs as the caller speaks, so much of it overlaps the utterance. The LLM’s reasoning is variable and depends on the model and the prompt. Text-to-speech first audio is the last stage and the most visible, because it is the moment the caller finally hears something back.
Why first audio dominates
First-audio time is the last thing between reasoning and sound. A model that starts speaking in under 100ms keeps the reply feeling immediate even when the earlier stages take their share. SpeechifyAI’s Simba 3.2 does exactly that while sitting at #1 on the Artificial Analysis TTS leaderboard, so the speed does not come at the cost of voice quality.
Frequently asked questions
What is voice agent latency?
What latency should a voice agent aim for?
Why does TTS first-byte latency matter so much?
Start building
Latency is the difference between a conversation and a wait. Here is the budget and the breakdown.