WebSocket TTS streaming for voice agents: LLM tokens in, speech out, measured in October 2026

Since 9 October 2026, SpeechifyAI’s text-to-speech API takes an LLM’s reply over a WebSocket as the tokens arrive, cuts it into sentences on the server and streams back audio, with word timestamps if you ask for them. Measured from Google Cloud us-east4 that day, the first word was audible 49 ms after the LLM finished its first sentence over the WebSocket, and 68 ms after it with one HTTP request per sentence, at the median (snapshot). Sending the whole reply in one request after the LLM’s last token kept the listener in silence about 2.1 seconds longer, nearly all of it the LLM still writing.

Counted from the first token, the first word was audible 449 ms in over the WebSocket and 468 ms in with one request per sentence, and 400 ms of each was our simulated LLM writing its first sentence at a token every 40 ms; no text-to-speech API can speak a sentence before it is written. The whole reply was audible 2,525 ms in, 2,440 ms of it the LLM writing up to its last token. The WebSocket started speaking 19 ms sooner than one HTTP request per sentence on a warm connection at the median, and 12 ms sooner at the 90th percentile. So the case for the WebSocket is less the milliseconds than the code you no longer write: no sentence splitter, no queue of requests, no player that reorders responses, and word timestamps on one timeline (snapshot).

What the numbers say

A segmenting transport cannot speak a sentence before the sentence exists. In this reply the first sentence was complete 400 ms after the first token, so both the WebSocket and the per-sentence requests started from the same floor, and the whole-reply request started from the last token, 2,440 ms in (snapshot).

From that floor, the first audio byte arrived 39 ms after the first sentence was complete over the WebSocket and 58 ms after it with per-sentence requests, and 75 ms after the last token for the whole reply, at the median (snapshot). The socket is authenticated and passes our load balancer’s security policy once, when it opens, where every HTTP request passes both again; that is the likeliest home of the difference, though this measurement does not split it.

The socket also costs something the table leaves out. It opened in 65 ms at the median and 86 ms at the 90th percentile from us-east4, before the first token, the way an agent opens it while its LLM is still working on the first token (snapshot). Open it only once the first token exists and that time adds to its first audio, which puts it behind a warm HTTP connection; open it with the LLM request and it costs nothing a caller hears.

The last audio arrived 15 ms apart on the two segmenting transports at the median and 12 ms apart at the 90th percentile, and in none of the 150 sessions did the player run out of audio before the reply ended (snapshot). Synthesis stayed ahead of playback on all three transports, so playback never paused to wait for audio.

When to use which transport

• The WebSocket, GET /v1/audio/stream/ws, when the text arrives in pieces, an LLM’s reply above all, and you would otherwise write the splitter, the request queue and the ordered player yourself.
• One HTTP request per sentence, POST /v1/audio/stream, when your framework already splits the reply into sentences, as the Speechify plugin for LiveKit Agents does, or when you need SSML: Simba 3.2 applies <break> pauses and <prosody rate> over HTTP, and the WebSocket reads SSML aloud as plain text.
• The whole text in one request when you have all of it before speaking starts, such as narration, or when one continuous delivery matters more than an early start, because a single synthesis carries its intonation across the whole text.

The streaming guide covers POST /v1/audio/stream.

Stream an LLM reply in Python

This is the example from the streaming text input guide, which has the full message reference, the rules for where a sentence ends and every way a session can end. It opens the socket from your server, with the API key in the Authorization header and the voice, model and output format in the query string for the whole session. It sends each piece of the reply as an input.text message the moment it exists, then input.close to have the rest spoken and end the session. Audio comes back as Base64 in speech.chunk events, with the word timestamps that speech_marks=true asks for, and the example prints each word’s timestamp and saves the audio to speech.pcm. Replace llm_reply with your LLM’s stream.

# pip install websockets
import asyncio
import base64
import json
import os
from urllib.parse import urlencode

from websockets.asyncio.client import connect

URL = "wss://api.speechify.ai/v1/audio/stream/ws?" + urlencode(
    {"voice_id": "geffen_32", "model": "simba-3.2", "output_format": "pcm_24000", "speech_marks": "true"}
)


async def llm_reply():
    # Stands in for your LLM's streamed reply.
    for token in ["Your", " order", " shipped", " this", " morning", ". It", " arrives", " on", " Thursday", "."]:
        yield token
        await asyncio.sleep(0.05)


async def send_reply(ws):
    async for token in llm_reply():
        await ws.send(json.dumps({"type": "input.text", "text": token}))
    await ws.send(json.dumps({"type": "input.close"}))


async def main():
    headers = {"Authorization": f"Bearer {os.environ['SPEECHIFY_API_KEY']}"}
    async with connect(URL, additional_headers=headers) as ws:
        sender = asyncio.create_task(send_reply(ws))
        with open("speech.pcm", "wb") as audio:
            async for frame in ws:
                event = json.loads(frame)
                if event["type"] == "speech.chunk":
                    if "audio" in event:
                        audio.write(base64.b64decode(event["audio"]))
                    for mark in event.get("speech_marks", []):
                        print(f"{mark['start_time']:>6} ms  {mark['value']}")
                elif event["type"] == "speech.done":
                    print(f"done ({event['reason']}): {event['billable_characters_count']} characters")
                    break
                elif event["type"] == "speech.error":
                    error = event["error"]
                    print(f"error {error['code']}: {error['message']} (request {event['request_id']})")
                    break
        sender.cancel()


asyncio.run(main())

Play the saved audio with ffplay:

ffplay -f s16le -ar 24000 -ac 1 speech.pcm

The same in TypeScript

// npm install ws
import { createWriteStream } from "node:fs";
import WebSocket from "ws";

const params = new URLSearchParams({
  voice_id: "geffen_32",
  model: "simba-3.2",
  output_format: "pcm_24000",
  speech_marks: "true",
});
const ws = new WebSocket(`wss://api.speechify.ai/v1/audio/stream/ws?${params}`, {
  headers: { Authorization: `Bearer ${process.env.SPEECHIFY_API_KEY}` },
});
const audio = createWriteStream("speech.pcm");

// Stands in for your LLM's streamed reply.
async function* llmReply() {
  for (const token of ["Your", " order", " shipped", " this", " morning", ". It", " arrives", " on", " Thursday", "."]) {
    yield token;
    await new Promise((resolve) => setTimeout(resolve, 50));
  }
}

ws.on("open", async () => {
  for await (const token of llmReply()) {
    ws.send(JSON.stringify({ type: "input.text", text: token }));
  }
  ws.send(JSON.stringify({ type: "input.close" }));
});

ws.on("message", (data) => {
  const event = JSON.parse(data.toString());
  switch (event.type) {
    case "speech.chunk":
      if (event.audio) audio.write(Buffer.from(event.audio, "base64"));
      for (const mark of event.speech_marks ?? []) console.log(`${mark.start_time} ms  ${mark.value}`);
      break;
    case "speech.done":
      console.log(`done (${event.reason}): ${event.billable_characters_count} characters`);
      break;
    case "speech.error":
      console.error(`error ${event.error.code}: ${event.error.message} (request ${event.request_id})`);
      break;
  }
});

// A refused upgrade is an HTTP response carrying the error envelope.
ws.on("unexpected-response", (request, response) => {
  let body = "";
  response.on("data", (chunk) => (body += chunk));
  response.on("end", () => {
    console.error(`HTTP ${response.statusCode}: ${body}`);
    request.destroy();
  });
});

ws.on("close", () => audio.end());

A setting the API does not accept, an unknown voice, an exhausted balance or a full rate or concurrency limit is refused before the upgrade, as an ordinary HTTP error with the standard error envelope, which is what the unexpected-response handler prints.

Limits to plan for

• Prosody restarts at each seam. Each segment is its own synthesis, so intonation and pacing start afresh between segments, and a pause can fall between two sentences that one request would have joined smoothly. Cartesia’s WebSocket lets you “send fragments as they arrive (for example from an LLM) and keep prosody across them”, read on 9 October 2026, and ours does not.
• Your server holds the key. The upgrade authenticates with Authorization: Bearer, which a browser cannot set on a WebSocket, and the key never goes in the URL, so connect from the server where your LLM runs.
• A session lasts at most 5 minutes. After 5 minutes it stops taking text, speaks what it has and ends with session_limit, and a session cannot be resumed, so open a new one to continue.
• A quiet session closes after 60 seconds. With nothing to speak and no message from you for 60 seconds, it ends with idle.
• One session holds one concurrent request. It holds one of your plan’s concurrent requests for as long as it is open, speaking or not (API limits).
• Each segment is a request. Each segment is billed as one POST /v1/audio/stream request for the characters it speaks, and each segment after the first uses one request from your rate limit.
• Plain text only. SSML sent over the socket is read aloud, not interpreted.
• The first sentence sets the floor. Nothing is spoken until the first sentence is followed by whitespace or you send input.flush, so a reply that opens with a long sentence starts later.

Cartesia, ElevenLabs and Deepgram also take text over a WebSocket, read from their docs on 9 October 2026. We timed their WebSockets and ours with one script that day, and the tables are in our guide to choosing a low-latency TTS API for voice agents.

How we measured

• Where and when. A Google Cloud VM in us-east4, in northern Virginia, the region our API is served from, ran all 156 sessions between 10:14 and 10:25 UTC on 9 October 2026 (snapshot).
• The reply. Each reply was four sentences of a support agent’s answer, 62 tokens of about 4 characters each, written by a simulated LLM at one token every 40 ms, so the first sentence was complete at 400 ms and the last token came at 2,440 ms. Every session’s reply carried its own order number, time and tracking digits, so no session could reuse a cached result from an earlier one.
• The clocks. The time after the text was ready starts when the first sentence is complete, or at the last token for the whole reply. Every other time starts when the first token exists.
• First audible. It is the moment a player that starts on the first audio byte and plays the segments in order reaches the first 10 ms window whose RMS level reaches 1% of full scale, the onset rule in Coval’s methodology.
• The HTTP runs. One HTTP/2 connection to the API was kept open for the whole run, so the measured requests reused a warm connection. The per-sentence run sent each sentence the moment the next one started, the server’s own rule, and played the responses in order.
• The WebSocket runs. Each session opened its socket before the first token, and the opening time is reported apart.
• The sample. The three transports ran interleaved in a rotating order, 50 measured sessions each after two warm-up rounds, with simba-3.2, the geffen_32 voice and 24 kHz PCM, and no session failed. Percentiles interpolate between the two nearest sessions.

Create a free API key and stream your own LLM’s reply into the socket.