Dynamic video narration using the Speechify Voice Cloning API

It’s 2026 and every TikTok video sounds the same. With Speechify, a regular text-to-speech request with us can be transformed by switching out one of our voices with your own, as easily as changing the voice_id to the one you just cloned.

This walks the native (no-SDK) version of the clone, use, delete lifecycle. The full recipe is in the cookbook at recipes/audio/typescript/native/voice-cloning, with a runnable end-to-end version in demos/voice-cloning-narration. You’ll need a Speechify API key to run any of it. Sign up here if you don’t have one yet.

What “voice cloning” actually does

You create a consent challenge, record the speaker reading its phrase, and POST that recording with an audio sample of the same person. The API returns a voice_id. Any future synthesis call with that voice_id produces audio in the cloned voice. The clone persists on your account until you delete it.

That’s it. Two calls, no fine-tuning step, no waiting queue, no separate “is the model ready” poll. The clone is usable on the next request.

The sample should be 10 to 30 seconds of clean speech. Single speaker, no music, minimal background noise. Phone-quality audio works but produces phone-quality output, so this is the one place where the input matters more than the prompt.

The bit that shocked me

I cloned my own voice as a smoke test for this post, expecting the usual uncanny-valley approximation. What came back was close enough that I’m posting both clips here so someone else can tell me if I’m imagining it.

Both samples below are me, in the sense that neither one is anyone else. One is a real microphone recording, the other is Speechify synthesizing text using a voice cloned from that same recording. I’m not going to tell you which is which, that would ruin the joke.

Cloning a voice with the REST API

Cloning takes two calls. POST /v1/voices/consent-challenges takes the speaker’s full_name and returns a phrase for them to read aloud, and POST /v1/voices, multipart form-data, takes name, gender, sample, consent_challenge_id and consent_recording. One gotcha the cookbook recipe gets right that everyone gets wrong the first time: when you pass a FormData to fetch, do NOT set Content-Type: multipart/form-data manually. fetch sets the boundary itself from the FormData instance. If you set the header yourself, your boundary value is missing or wrong and the upload fails with a non-obvious error. I’ve watched this trip up three different engineers on three different teams, including past-me.

import "dotenv/config";
import fs from "node:fs";
import path from "node:path";
import readline from "node:readline/promises";

const token = process.env.SPEECHIFY_API_KEY;
if (!token) throw new Error("Set SPEECHIFY_API_KEY");

const BASE = "https://api.speechify.ai";
const samplePath = path.resolve(import.meta.dirname, "fixtures/my-voice.wav");
const consentPath = path.resolve(import.meta.dirname, "fixtures/consent.wav");

const challengeRes = await fetch(`${BASE}/v1/voices/consent-challenges`, {
  method: "POST",
  headers: { Authorization: `Bearer ${token}`, "Content-Type": "application/json" },
  body: JSON.stringify({ full_name: "Jane Doe" }),
});
if (!challengeRes.ok) {
  throw new Error(`POST /v1/voices/consent-challenges → ${challengeRes.status} ${await challengeRes.text()}`);
}
const challenge = await challengeRes.json() as { id: string; phrase: string; expires_at: string };

console.log(`Read this aloud, exactly as written:\n\n  ${challenge.phrase}\n`);
const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
await rl.question(`Save the recording to ${consentPath}, then press Enter (before ${challenge.expires_at}).`);
rl.close();

const form = new FormData();
form.append("name", "narrator-v1");
form.append("gender", "male");
form.append("consent_challenge_id", challenge.id);

const sampleBytes = fs.readFileSync(samplePath);
form.append("sample", new Blob([sampleBytes], { type: "audio/wav" }), "my-voice.wav");
const consentBytes = fs.readFileSync(consentPath);
form.append("consent_recording", new Blob([consentBytes], { type: "audio/wav" }), "consent.wav");

const createRes = await fetch(`${BASE}/v1/voices`, {
  method: "POST",
  headers: { Authorization: `Bearer ${token}` }, // no Content-Type here
  body: form,
});

if (!createRes.ok) {
  if (createRes.status === 402) {
    console.error("Voice cloning isn't included in your current Speechify plan.");
    console.error("Upgrade at https://speechify.ai/pricing");
    process.exit(1);
  }
  throw new Error(`POST /v1/voices → ${createRes.status} ${await createRes.text()}`);
}

const voice = await createRes.json() as { id: string; display_name: string; type: string };
console.log(`Created voice ${voice.id}`);

That 402 path is real, I hit it on my own dev account the first time I ran this. The response envelope is the standard Speechify shape:

{
  "error": {
    "code": "voice_cloning_not_included",
    "message": "current billing plan does not have access to voice cloning"
  },
  "request_id": "993cae7e..."
}

voice_cloning_not_included is the error code to branch on if you’re shipping this to customers whose plan may or may not include cloning. Don’t catch the generic 402.

Consent isn’t a footer note

Consent is verified, not asserted. Voice cloning is the API in the Speechify catalogue with the most obvious abuse vector, and the consent recording is what differentiates legitimate use from “I scraped a podcast.” Speechify transcribes it, checks it against the phrase it issued, checks the speaker against the sample, and keeps it as the consent record for the voice. Three refusals share 422, so branch on the error code:

• consent_phrase_mismatch: the recording doesn’t say the phrase. Show it exactly as returned and record again.
• consent_speaker_mismatch: the person reading the phrase isn’t the person in the sample. The speaker consenting has to be the speaker being cloned.
• consent_recording_unusable: silence, too little or too much speech, or an unreadable file. Keep it between 5 and 30 seconds.

A challenge is single use and expires at expires_at; an expired or already used one returns 409 (consent_challenge_expired, consent_challenge_already_used). A refused create spends it, so every retry starts with a new challenge and a new phrase. Create the challenge when the speaker is ready to record, not at the start of a job. For your demos, clone your own voice and read the phrase yourself. For your product, the person whose voice it is records the phrase themselves.

Synthesizing with the cloned voice

Once you have a voice_id, the synthesis call is identical to the default-voice version. Same endpoint, same body shape, the cloned id slots in where sabrina or any other catalogue voice would go:

const speechRes = await fetch(`${BASE}/v1/audio/speech`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${token}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    input: "Hello from a voice cloned with the Speechify API.",
    voice_id: voice.id, // the id you just got back from /v1/voices
    audio_format: "mp3",
    model: "simba-3.0",
  }),
});

const speech = await speechRes.json() as { audio_data: string };
fs.writeFileSync("narration.mp3", Buffer.from(speech.audio_data, "base64"));

If you read the Node.js TTS post, this is the same request you’ve already made, just with a different value for one field. The cloned voice acts identical to a catalogue voice from this point on, which means streaming, speech marks (see the captions post if you want to wire those up), and SSML all work exactly as they do for any other voice.

Cleaning up

Cloned voices stick around on your account until you delete them. For one-off video narration jobs, the lifecycle is usually clone, synthesize, delete:

try {
  // ... synthesis ...
} finally {
  const delRes = await fetch(
    `${BASE}/v1/voices/${encodeURIComponent(voice.id)}`,
    { method: "DELETE", headers: { Authorization: `Bearer ${token}` } },
  );
  if (!delRes.ok) {
    console.error(`Cleanup failed: ${delRes.status} ${await delRes.text()}`);
  }
}

The finally matters. If your synthesis throws, you still want the voice gone, otherwise you’ve got an orphan on your account every time a job fails halfway. voices.list() is eventually consistent so a just-deleted voice may show up for a second or two on the next list call, but the delete itself is immediate. A subsequent delete returns 404.

For long-running narrator identities (a podcast host, a recurring character), keep the voice and reuse the voice_id. Don’t re-clone every time, it costs an upload and adds latency.

Stitching audio into a video

Once you’ve got narration.mp3, the rest is ffmpeg. If your video is silent footage plus the narration:

ffmpeg -i video.mp4 -i narration.mp3 \
  -c:v copy -c:a aac -shortest \
  output.mp4

-c:v copy keeps your video stream untouched (no re-encode, no quality loss). -c:a aac re-encodes the narration to AAC, which is the audio codec MP4 wants. -shortest trims the output to whichever stream ends first, which is how you avoid trailing silence if the narration is shorter than the visuals.

For a static-image video (talking head replacement, podcast cover art, audiobook splash):

ffmpeg -loop 1 -i cover.png -i narration.mp3 \
  -c:v libx264 -tune stillimage -c:a aac \
  -pix_fmt yuv420p -shortest \
  output.mp4

That’s enough for YouTube Shorts, podcast clips, audiobook samples, and anywhere else you’d otherwise be paying for a stock TTS voice a million other pipelines are already using.

FAQ

Is voice cloning included on every Speechify plan?

No. Voice cloning is plan-gated. The API returns 402 voice_cloning_not_included if your current plan does not include it. Pricing and which tiers include cloning live on the Speechify pricing page. Branch on the voice_cloning_not_included error code (not the generic 402 status) so your customer-facing error message is precise.

How long should the audio sample be?

Between 10 and 30 seconds of clean speech, single speaker, minimal background noise. Less than 10 seconds and the clone is hit-or-miss on prosody. More than 30 seconds adds upload time without improving the output meaningfully. Phone-recorded audio works but produces phone-quality clones, so for a polished narrator voice the input has to be polished too.

Can I use a cloned voice with streaming and speech marks?

Yes. Once /v1/voices gives you back a voice_id, it acts identically to any catalogue voice. You can stream it through /v1/audio/stream for low time-to-first-byte playback, and the non-streaming /v1/audio/speech endpoint returns word-level speech marks alongside the audio for caption generation. Both work without any additional clone-specific configuration. The full parameter surface for both endpoints lives in the Speechify docs.

How does consent work when cloning a voice?

The speaker reads a phrase Speechify issues. Create a challenge with POST /v1/voices/consent-challenges and the speaker’s full_name, record them reading the returned phrase, and send that recording as consent_recording with the challenge’s id as consent_challenge_id on POST /v1/voices. The recording has to be the same person as the sample, 5 to 30 seconds long. The old consent field, a name and email as a JSON string, was switched off on 23 September 2026: outside a few accounts on an existing agreement, it now returns 400 consent_verification_required.