Controlling Emotion and Timing in TTS with SSML

SSML emotion TTS lets you send delivery instructions with the text: emotion, pauses, pace, pitch, emphasis, and pronunciation fixes. With Speechify, those instructions go in the same input field you already use for POST /v1/audio/speech, so the request shape stays familiar and the spoken result stops sounding like one flat paragraph.

The runnable demo for this post is speechify-ai-demos/ssml-emotion-tts. I ran it end to end against the live Speechify API with voice_id: "george", model: "simba-english", and the TypeScript SDK. It wrote a real output/ssml-emotion.mp3 file, 172,076 bytes in my run. You’ll need a Speechify API key to run it yourself. Create one here if you do not have one yet.

What does SSML emotion TTS control?

SSML is Speech Synthesis Markup Language, an XML format for telling a speech model how text should be spoken. In Speechify’s TTS API, SSML can control five parts of delivery:

• emotion with <speechify:style emotion="...">
• silence with <break time="500ms" />
• pacing, pitch, and loudness with <prosody>
• stress with <emphasis>
• pronunciation substitutions with <sub alias="...">

That means your app can make a status update sound warm, slow down a safety instruction, pause before a number, stress one word, and pronounce an acronym correctly without splitting the script into separate audio jobs.

The full tag reference lives in the Speechify SSML docs , and the emotion presets are covered in the emotion control guide .

The one XML rule you cannot skip

The whole input has to be one SSML document with one <speak> root. Everything spoken goes inside that root.

<speak>Everything you want spoken goes inside one speak element.</speak>

SSML is XML, so reserved characters in interpolated text need escaping. & becomes &, < becomes <, and > becomes >. This matters most when part of the script comes from a CMS, an LLM, or customer input.

function escapeXml(value: string): string {
  return value
    .replaceAll("&", "&")
    .replaceAll("<", "&lt;")
    .replaceAll(">", "&gt;")
    .replaceAll('"', "&quot;")
    .replaceAll("'", "&apos;");
}

If you are already making the basic Node.js TTS request, the switch is small. Plain text becomes a <speak> document. The endpoint, auth header, output format, and base64 decoding stay the same.

Setting emotion without changing endpoints

Speechify supports 13 emotion values in SSML: angry, cheerful, sad, terrified, relaxed, fearful, surprised, calm, assertive, energetic, warm, direct, and bright.

Wrap a sentence, clause, or whole paragraph in <speechify:style> and set the emotion attribute:

<speak>
  <speechify:style emotion="warm">Welcome to the production incident update.</speechify:style>
  <break time="500ms" />
  <speechify:style emotion="assertive">Do not restart the workers unless the backlog climbs again.</speechify:style>
</speak>

The useful part is switching emotion inside one request. A support bot can sound calm at the start of a message, direct when it gives the next step, and warm again when it confirms the customer is done. A training video can keep the narrator friendly, then move into an assertive warning when the script reaches a safety step.

If you are using a cloned voice for dynamic narration, the same idea applies. Clone the voice as shown in the voice cloning lifecycle post, then send SSML in the later synthesis call.

Timing is where scripts start to feel human

Most flat TTS problems are timing problems. Text has punctuation, but punctuation is a weak signal for speech. The listener hears the difference between a comma, a half-second beat, and a slower explanation.

Use <break> when you want silence for a known duration:

<speak>
  The migration starts in five minutes.
  <break time="500ms" />
  Keep the dashboard open until the final shard clears.
</speak>

Use <prosody> when you want a span of speech delivered differently:

<speak>
  <prosody rate="slow" pitch="low">
    The queue is draining, but keep monitoring error rates for the next ten minutes.
  </prosody>
</speak>

For real-time playback you may still want streaming, especially if the script is generated on demand. The streaming setup is covered in Streaming TTS in Python with Speechify. For pre-rendered clips, product walkthroughs, onboarding videos, and IVR prompts, the non-streaming speech endpoint is usually the simpler fit because you get one complete MP3 back.

Emphasis and pronunciation fixes belong in the script

<emphasis> is for the word you would lean on if you were speaking out loud:

<speak>This is <emphasis level="strong">critical</emphasis> for customer playback.</speak>

<sub> is for text that should be written one way and spoken another way:

<speak>Status page says <sub alias="all systems operational">ASO</sub> once the final region clears.</speak>

That second tag is the quiet workhorse. Product names, ticker symbols, internal acronyms, chemical names, and initialisms usually need pronunciation rules. Keeping the written text intact also helps if you pair audio with speech marks for captions, like the captions and speech marks post does.

Full TypeScript demo

The demo uses @speechify/api@^3.0.1 and the SDK constructor shape from the current package:

import "dotenv/config";
import fs from "node:fs";
import path from "node:path";
import { SpeechifyClient, SpeechifyError } from "@speechify/api";

const token = process.env.SPEECHIFY_API_KEY;
if (!token) {
  throw new Error("Set SPEECHIFY_API_KEY (copy .env.example to .env).");
}

const client = new SpeechifyClient({ token });

const ssml = `<speak>
  <speechify:style emotion="warm">Welcome to the production incident update.</speechify:style>
  <break time="500ms" />
  <prosody rate="slow" pitch="low">The queue is draining, but keep monitoring error rates for the next ten minutes.</prosody>
  <break time="300ms" />
  <speechify:style emotion="assertive">Do not restart the workers unless the backlog climbs again.</speechify:style>
  <break time="400ms" />
  This is <emphasis level="strong">critical</emphasis> for customer playback.
  <break time="300ms" />
  Status page says <sub alias="all systems operational">ASO</sub> once the final region clears.
</speak>`;

try {
  const response = await client.audio.speech({
    input: ssml,
    voice_id: "george",
    audio_format: "mp3",
    model: "simba-english",
  });

  const outDir = path.resolve("output");
  fs.mkdirSync(outDir, { recursive: true });

  const outPath = path.join(outDir, "ssml-emotion.mp3");
  fs.writeFileSync(outPath, Buffer.from(response.audio_data, "base64"));

  console.log(`Wrote output/ssml-emotion.mp3 (${response.billable_characters_count} billable characters).`);
} catch (err) {
  if (err instanceof SpeechifyError) {
    console.error(err.message);
    process.exit(1);
  }
  throw err;
}

Run it from the demo folder:

cp .env.example .env
npm install
NODE_EXTRA_CA_CERTS=/etc/ssl/cert.pem SPEECHIFY_API_KEY=$SPEECHIFY_API_KEY npm start

The TLS environment variable is only there for local Node setups that need the system certificate bundle. The API call itself is still a normal SDK call to Speechify.

The model and voice pair matter. This demo uses voice_id: "george" with model: "simba-english", which is the same pairing used by the cookbook recipe. If you are comparing model quality across use cases, the Artificial Analysis leaderboard note is a useful reference point, but test your own scripts. A leaderboard sentence is not the same as your production prompt.

When should you use SSML instead of editing the text?

Use SSML when the spoken form needs information that does not belong in the written text. Pauses, emotional tone, pronunciation aliases, and emphasis are delivery instructions. They should not pollute the script that your app stores, displays, localizes, or uses for captions.

Plain text is still fine for short, low-stakes clips: a generated confirmation, a one-line notification, or a quick prototype. Once the audio carries product meaning, SSML is worth adding early. It is cheaper to put timing into the script than to regenerate clips until one accidentally sounds right.

FAQ

Do I need a separate Speechify endpoint for SSML emotion TTS?

No. Send the SSML document as input to the same POST /v1/audio/speech request. The document must have one <speak> root, and the rest of the body stays familiar: voice_id, audio_format, and model. The demo uses the TypeScript SDK, but the native REST request uses the same JSON body.

Which Speechify emotions can I use in SSML?

There are 13, listed in full earlier in this post, from cheerful and warm to assertive and terrified. You set one with the emotion attribute on <speechify:style>, and you can use more than one style tag in a single document so the voice changes register as the script changes.

Why did my SSML request fail to parse?

Check the XML first. The whole document needs one <speak> root, and text inside it needs XML escaping for reserved characters like &, <, and >. This shows up when you interpolate user input or LLM output directly into an SSML string. Escape the text before you assemble the document.

Can I combine SSML with speech marks or cloned voices?

Yes. SSML changes synthesis delivery, not the endpoint family. You can use it with catalogue voices or cloned voices, and the non-streaming speech endpoint can still return speech marks alongside audio. If your final product needs captions, keep the visible script and pronunciation aliases aligned so the text on screen still makes sense.

What should I test before shipping SSML audio?

Run the exact SSML through the live API, listen to the generated MP3, and keep the output artifact next to the demo or test fixture. Validate at least one pause, one emotion change, one prosody span, one emphasized word, and one pronunciation alias. SSML bugs are easy to miss if you only inspect the XML.