Build a LiveKit voice agent with Speechify TTS

LiveKit's Speechify plugins, for Python and Node.js, add Speechify TTS to an AgentSession in one line. The Python plugin streams each sentence: 98 ms median to first audible audio from US East on 12 Oct 2026.

Engineering and research · SpeechifyAI
Updated 9 min read

You add Speechify TTS to a LiveKit voice agent with one line: install LiveKit’s Python plugin and pass speechify.TTS(...) as the tts of your AgentSession. The plugin streams, so audio starts while the rest of the sentence is still being synthesized. On 12 October 2026 we measured 98 ms from handing it a sentence to its first audible audio, at the median, from a server in US East.

This guide wires a complete agent with Deepgram for speech-to-text, OpenAI for the language model and speechify.TTS(voice_id="dominic_32", model="simba-3.2") for speech, then shows you how to measure first audio from where your own agent runs. It is written in Python. LiveKit’s Node.js framework has its own Speechify plugin since 9 October 2026, and the Node.js section shows it. You need a Speechify API key, a Deepgram key and an OpenAI key. Get a Speechify key here if you do not have one yet.

How the plugin streams

A LiveKit voice agent is a loop: the user speaks, STT turns it into text, the LLM writes a reply, and TTS speaks it. Each stage is a provider object on the AgentSession, and LiveKit runs the conversation around them.

LiveKit AgentSession

Speechify is the TTS provider in the voice pipeline

LiveKit handles the realtime room and turn loop. The Python agent passes Deepgram, OpenAI, and Speechify providers into one session.

  1. STTDeepgram listensUser audio becomes text inside the LiveKit session.deepgram.STT(model="nova-3")
  2. LLMOpenAI repliesThe agent turns the transcript into a short conversational response.openai.LLM(...)
  3. TTSSpeechify speaksThe official LiveKit plugin calls Speechify with the voice and model you choose.speechify.TTS(...)

The Speechify key stays in the Python environment as SPEECHIFY_API_KEY. The browser or room participant never receives it.

The Speechify stage splits the LLM’s reply into sentences as the tokens arrive. Each sentence becomes one request to Speechify’s /v1/audio/stream/with-timestamps endpoint, and the plugin hands LiveKit each audio chunk the moment it lands, with the word timestamps for that chunk riding on the same stream. So the first word plays after the first complete sentence and one streaming request, not after the whole reply. Since version 1.8.6 every sentence reuses one open connection, and AgentSession calls the plugin’s prewarm() when the agent starts, so even the first reply does not wait for a connection to open.

Two things follow from that shape. First, the TTS number below is only one part of what a caller waits for: the turn also includes end-of-speech detection, the transcript, and the LLM’s first sentence. Second, the word timestamps are free: the plugin reports them to LiveKit, so use_tts_aligned_transcript=True on the session syncs the transcript to the audio word by word. Word-level timestamps in a LiveKit voice agent builds captions on them.

Install the Python packages

Create a fresh Python environment and install LiveKit Agents, the Speechify plugin, and the STT and LLM plugins this agent uses:

python3 -m venv .venv
source .venv/bin/activate
pip install "livekit-agents[codecs]>=1.8.6" \
  "livekit-plugins-speechify>=1.8.6" \
  "livekit-plugins-deepgram>=1.8.6" \
  "livekit-plugins-openai>=1.8.6" \
  python-dotenv

livekit-plugins-speechify is authored and published by LiveKit. Version 1.8.6, released on 9 October 2026, is the first to reuse one connection across sentences, stream synthesize() and open the connection early with prewarm(), so pin at least that. On 12 October 2026 that command resolved livekit-agents 1.8.6 and livekit-plugins-speechify 1.8.6, and every snippet in this guide was checked against those versions.

Configure the environment

Put your keys in .env:

SPEECHIFY_API_KEY=your_speechify_api_key
DEEPGRAM_API_KEY=your_deepgram_api_key
OPENAI_API_KEY=your_openai_api_key

# Only needed when you run against a real LiveKit room (dev or start):
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=your_livekit_api_key
LIVEKIT_API_SECRET=your_livekit_api_secret

The plugin reads SPEECHIFY_API_KEY unless you pass api_key= to speechify.TTS(...) yourself. Keep it server-side in the Python agent, and never hand it to a browser participant.

Wire Speechify into AgentSession

Here is the full agent, also in the runnable livekit-agent-speechify-python demo:

from dotenv import load_dotenv

from livekit import agents
from livekit.agents import Agent, AgentServer, AgentSession
from livekit.plugins import deepgram, openai, speechify

load_dotenv()


class Assistant(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions=(
                "You are a helpful voice assistant speaking with a Speechify voice. "
                "Keep replies short, clear, and conversational."
            )
        )


server = AgentServer()


@server.rtc_session(agent_name="speechify-tts-demo")
async def entrypoint(ctx: agents.JobContext) -> None:
    session = AgentSession(
        stt=deepgram.STT(model="nova-3"),
        llm=openai.LLM(model="gpt-4o-mini"),
        tts=speechify.TTS(voice_id="dominic_32", model="simba-3.2"),
    )

    await session.start(room=ctx.room, agent=Assistant())
    await session.generate_reply(
        instructions=(
            "Greet the user and mention that your voice "
            "is powered by Speechify TTS."
        )
    )


if __name__ == "__main__":
    agents.cli.run_app(server)

Save it as agent.py. The tts= line is the whole integration. simba-3.2 is the streaming model for English; for German, Spanish, French, Italian or Brazilian Portuguese, pass a voice for that language with model="simba-3.0".

Measure first audio before you open a room

The number a caller feels depends on where your agent runs, so measure it there before you tune anything else. This script needs only SPEECHIFY_API_KEY, no LiveKit room and no microphone. It hands the plugin one sentence at a time, the way AgentSession does, and times when the first audible sound could play: the first 10 ms of audio louder than -45 dBFS, and never before the frame that carries it has arrived.

import array
import asyncio
import math
import statistics
import time

from dotenv import load_dotenv
from livekit.plugins import speechify

RATE = 24000
WINDOW = RATE // 100  # 10 ms
AUDIBLE = 32768 * 10 ** (-45 / 20)  # -45 dBFS


async def first_audible_ms(tts: speechify.TTS, text: str) -> float:
    stream = tts.stream()
    start = time.perf_counter()
    stream.push_text(text)
    stream.end_input()
    first = audible = None
    samples = array.array("h")
    async for event in stream:
        now = time.perf_counter()
        first = first or now
        if audible is None:
            samples.extend(event.frame.data)
            for i in range(0, len(samples) - WINDOW + 1, WINDOW):
                window = samples[i : i + WINDOW]
                if math.sqrt(sum(s * s for s in window) / WINDOW) > AUDIBLE:
                    audible = max(first + i / RATE, now)
                    break
    await stream.aclose()
    return (audible - start) * 1000


async def main() -> None:
    load_dotenv()
    tts = speechify.TTS(voice_id="dominic_32", model="simba-3.2")
    await first_audible_ms(tts, "Warming up.")  # not counted

    times = []
    for n in range(20):
        text = f"Your order {1000 + n} shipped today and arrives Thursday."
        times.append(await first_audible_ms(tts, text))

    deciles = statistics.quantiles(times, n=10)
    print(f"first audible p50 {deciles[4]:.0f} ms, p90 {deciles[8]:.0f} ms")
    await tts.aclose()


asyncio.run(main())

Run from our development server in Zurich on 12 October 2026, it printed:

first audible p50 180 ms, p90 265 ms

Zurich is a long way from the API, which serves from US East, so most of that is the Atlantic. Run it from the region your agent is deployed in.

What we measured

We ran the same measurement from a fresh virtual machine in each of two Google Cloud regions on 12 October 2026, with simba-3.2 and dominic_32. Version 1.8.6 and the previous release we measured, 1.8.3, ran side by side in the same run. Every request was a different sentence of about 90 characters, each cell is 25 requests, and the cells ran interleaved so that time of day hit them alike. “First reply” is a new TTS object, as in the first turn of a new call, timed 1 s after prewarm(); “later turn” reuses one. The last row is a bare POST /v1/audio/stream on an open connection, the floor for anything built on the API.

First audible audio, p50 / p90us-east4, Virginiaeurope-west6, Zurich
1.8.6, first reply98 / 143 ms194 / 217 ms
1.8.6, later turn104 / 136 ms196 / 230 ms
1.8.3, first reply108 / 145 ms226 / 252 ms
1.8.3, later turn109 / 153 ms228 / 268 ms
Bare API78 / 103 ms172 / 198 ms
Time from handing the plugin a sentence to when its first audible audio could play, 25 requests per cell, 12 October 2026.

From Zurich, 1.8.6 is about 30 ms faster than 1.8.3 at the median on both kinds of reply. From US East the medians moved by 10 ms on the first reply and 5 ms on a later turn, which is small next to the spread between requests. The change underneath is the connection: on a later turn 1.8.3 opened a new one for every sentence, and 1.8.6 opened none.

To the first audio frame, 1.8.6 was within 5 ms of the bare API at the median from both regions. The rest of the gap comes after the first frame: through the plugin’s endpoint the median first audible audio came 26 to 36 ms after the median first frame, against 5 to 10 ms on the bare /v1/audio/stream.

synthesize() changed the most. In 1.8.3 it made one non-streamed request for the whole text, and its first audible audio waited 1,626 ms at the median from us-east4 for the same sentences. Since 1.8.6 it streams like stream(), and it took 95 ms. AgentSession calls stream(), so an agent never took the slow path, but audio you render ahead of time with synthesize() now starts as quickly.

The first version of this guide, on 28 September 2026, measured 1.8.3 the same way from us-east4 and quoted the first audio frame, 82 ms at the median; first audible audio in that run was 101 ms.

Run the LiveKit agent

With .env filled in, start local console mode:

python agent.py console

Console mode runs against your microphone and speakers with no room, which keeps the loop local while you tune prompts, voice choice and interruption behavior.

To connect the agent to a real LiveKit project, use development mode:

python agent.py dev

For production, use LiveKit’s deployment flow or run python agent.py start. At that point you are operating a realtime service: rotate keys, watch model errors, and measure the whole turn from the caller’s last word to your agent’s first audible word. To size your Speechify plan for many calls at once, read TTS API concurrency and rate limits by plan.

The same agent in Node.js

LiveKit publishes the plugin for its Node.js framework too: @livekit/agents-plugin-speechify, on npm since 9 October 2026. It streams each sentence over the same endpoint with word timestamps, reuses one connection, and has the same prewarm(). Install it next to LiveKit Agents and the STT and LLM plugins:

npm install @livekit/agents @livekit/agents-plugin-speechify \
  @livekit/agents-plugin-deepgram @livekit/agents-plugin-openai

The session takes the same three providers, and the tts: line is again the whole integration:

import { voice } from '@livekit/agents';
import * as deepgram from '@livekit/agents-plugin-deepgram';
import * as openai from '@livekit/agents-plugin-openai';
import * as speechify from '@livekit/agents-plugin-speechify';

const session = new voice.AgentSession({
  stt: new deepgram.STT({ model: 'nova-3' }),
  llm: new openai.LLM({ model: 'gpt-4o-mini' }),
  tts: new speechify.TTS({ voiceId: 'dominic_32', model: 'simba-3.2' }),
});

The LiveKit page in the Speechify docs has the complete Node.js agent and a smoke test, both run against version 1.9.2. The numbers in this guide are for the Python plugin.

Choosing the Speechify voice

This demo uses dominic_32 with simba-3.2. simba-3.2 serves every English voice in the catalogue, cloned voices included, and the _32 voices are its built-in examples. Pass another voice_id whenever you want a different speaker, and check the voice’s models list on the /v1/voices endpoint to confirm the pairing.

For how the first-audio numbers compare across providers, read how to choose a low-latency TTS API for voice agents in 2026. For how providers compare on blind listening and price, read TTS APIs compared by Elo and price and what TTS APIs cost at volume. For the streaming API underneath the plugin, read Streaming TTS in Python with Speechify. If you use Deepgram’s Voice Agent instead of LiveKit, Add a better voice to Deepgram’s Voice Agent with Speechify is the same idea on that stack. The Speechify docs and LiveKit’s Speechify plugin guide cover the rest of the surface.

FAQ

Common questions

Does the LiveKit Speechify plugin stream audio?
Yes. With simba-3.2 or simba-3.0, the plugin's stream() sends each sentence to Speechify's /v1/audio/stream/with-timestamps endpoint and hands LiveKit audio frames as they arrive, with word timestamps on the same stream. The agent's reply is split into sentences first, so it is one streaming request per sentence. Since livekit-plugins-speechify 1.8.6 those requests share one open connection, and synthesize() streams too.
How fast is first audio through the LiveKit Speechify plugin?
On 12 October 2026 we measured 98 ms at the median and 143 ms at the 90th percentile from a server in Google Cloud us-east4, from handing livekit-plugins-speechify 1.8.6 a sentence to its first audible audio, on the first reply of a new session with simba-3.2, over 25 requests. From Zurich it was 194 ms at the median, because the API serves from US East.
Is there a Node.js LiveKit Speechify plugin?
Yes. Since 9 October 2026 LiveKit publishes @livekit/agents-plugin-speechify on npm for its Node.js Agents framework. Pass new speechify.TTS({ voiceId: 'dominic_32', model: 'simba-3.2' }) as the tts of a voice.AgentSession; like the Python plugin, it streams each sentence with word timestamps and has prewarm(). The LiveKit page in the Speechify docs has a complete Node.js agent and a smoke test.
Which environment variable does the LiveKit Speechify plugin read?
SPEECHIFY_API_KEY, unless you pass api_key= to speechify.TTS(...) yourself, or apiKey in the Node.js plugin. Keep the key server-side in the agent and never hand it to a browser participant.
Which keys do I need to run the full LiveKit agent?
SPEECHIFY_API_KEY for TTS, DEEPGRAM_API_KEY for speech-to-text and OPENAI_API_KEY for the LLM. LIVEKIT_URL, LIVEKIT_API_KEY and LIVEKIT_API_SECRET are only needed when you run against a real LiveKit room with the dev or start commands.

Privacy preferences

Choose what we may store on this device. You can change this at any time from the footer.

Strictly necessary

Sign-in, security, load balancing, and remembering your privacy choices. These cannot be switched off.

Always on

Analytics

How the site is used in aggregate - which pages get read, where people get stuck - so we can improve it.

Marketing

Measures which campaigns bring people here, and lets us show relevant ads on other platforms.