6.5 Voice: Cloning and Synthesis
For a persona who speaks, the voice is more durable than the face. Chapter 1.6's survey found that the enduring examples in this category are built on character continuity, and for a speaking persona the voice carries most of it. A listener recognizes a voice across a bad connection; a viewer does not recognize a face across a camera change.
Voice is also the cheapest layer in the video stack. A minute of generated speech costs cents where a minute of generated video costs dollars, and the ordering rule in this chapter is what keeps both cheap.
Two routes
Producing new speech from a sample of a real voice; anchoring to a fixed artifact is why it beats voice design.. You supply a sample of a real voice and the model produces new speech in it.
Describing a voice and having a model create one with no source recording: no consent problem, less consistency.. You describe a voice and the model creates one, with no source recording.
Why cloning is usually the right starting point, even though it involves a real person's voice: the consistency problem. A designed voice is a description, and regenerating it later reproduces something similar rather than identical. A cloned voice is anchored to a fixed artifact, and every generation comes back to it. This is the same argument as the The versioned plate of the persona's face and body that every generation is scored against. in 5.1 — an anchor beats a description.
The consent rule applies in full here. A clone is a real person's voice. Chapter 11.4 governs likeness and data, and it applies to audio exactly as it does to a face: written consent, a defined scope, no minors, and no clones of public figures. A voice clone without consent is a legal problem with a person attached, and it is not a gray area.
The alternative that avoids the issue entirely: hire a voice actor for a recording session that explicitly licenses synthesis, or design a voice from scratch and accept the reduced consistency. Both are legitimate; both are decisions to make deliberately rather than by accident.
Cost tiers
[AS OF 2026-10] [VENDOR CLAIM] Published pricing, which changes frequently. Verify before budgeting.
| Tier | Rough cost | Volume | Use for |
|---|---|---|---|
| Entry subscription | ~$6/month | ~30k characters | A persona posting a few times a week |
| Mid subscription | ~$22/month | ~120k characters | Several posts a day, or multiple languages |
| Pro subscription | ~$99/month | ~600k characters | Multi-language publishing at scale |
| Business subscription | ~$990/month | ~6M characters | Nothing a solo operator should need |
| Pay-per-use, standard | ~$60 per million characters | Metered | Bursty production |
| Pay-per-use, high definition | ~$100 per million characters | Metered | Hero pieces only |
| Clone creation | ~$3 per voice | One-off | Where cloning is supported at that tier |
Translating that into content. A thirty-second spoken post is roughly seventy to ninety words, approximately five hundred characters. The entry tier's 30k characters is therefore fifty to sixty spoken posts per month — comfortably more than chapter 2.6's cadence, at $6.
The conclusion worth stating plainly: voice is not a cost problem at any realistic solo-operator volume. The tier decisions that matter are quality and language coverage, not quota. An operator who optimizes voice spending has optimized the smallest number in the pipeline.
Clone quality: what determines it
Four factors, in order of impact. [UNVERIFIED] Reported by practitioners; the mechanisms are plausible and the magnitudes are not measured.
Source material quality. Clean recording, no background noise, no music, consistent microphone distance, no clipping. A twenty-minute clean recording outperforms two hours of phone audio with crosstalk.
Source material variety. Emphatic and calm, fast and slow, questions and statements. A clone built from one emotional register reproduces that register and struggles elsewhere — the voice equivalent of 4.4's The adapter reproducing whichever expression or pose dominated its dataset; a concentration failure, not a prompt failure..
Source material length. Minutes are enough for a usable clone; more improves stability. Diminishing returns after roughly an hour.
Consent and legality of the source. Not a quality factor, listed here because it is decided at the same moment. The consent comes first, and it is documented, scoped, and dated.
Emotional drift, and how to test
The failure mode specific to voice: a clone that reads neutral text well and loses its identity when the text demands emotion.
How it manifests. Anger, excitement, or sadness push the model toward a generic expressive register, and the persona's voice partially disappears. The listener cannot name what changed, but the character sounds less like themselves in exactly the posts where voice matters most.
The test. A five-line script: a neutral statement, a question, an exclamation, a soft/quiet line, and a joke with a specific timing beat. Generate all five, listen in sequence, and ask whether all five are the same person. This is the voice equivalent of 4.6's battery, and it takes five minutes.
The mitigations: use the clone's source variety to cover the registers you plan to use; keep emotional lines short; and where a line requires an extreme register the clone cannot hold, either rewrite the line or record that line with the licensed actor instead. Splitting the voice between a clone and a human is entirely legitimate and mostly undetectable, because listeners do not compare across a session.
The ordering rule
Voice first, then video, then Matching synthesized speech to a face; priced per minute and the most retried line in the cost sheet.. Never the reverse. And never let the video model generate the dialogue.
Two separate rules, both about drift.
Never let a video model generate dialogue, even where it offers native audio. [UNVERIFIED] Practitioners report the generated speech drifting in pitch and identity within a few seconds, since audio generation inside a video model has no persistent speaker reference and no anchor. Chapter 6.2 lists native audio as a feature to decline.
Generate voice first because the voice track defines the timing that everything else conforms to. Once the audio exists, its length determines the A CLIP-style text encoder that responds best to comma-separated fragments; used by older model families. length, its pauses determine the cut points, and its emphasis determines the gesture timing. Generating video first and fitting audio to it means re-generating video every time a line is adjusted — the expensive direction.
This ordering is why chapter 6.4's mismatch rule is cheap: mismatches are caught before assembly, while changing the video still means re-generating a five-second clip rather than re-editing a finished piece.
Multi-language, from one clone
One clone can speak many languages. [UNVERIFIED] This is one of the genuine advantages of a synthetic persona over a human creator, and it comes with three losses worth knowing.
What is preserved. Timbre, essentially — the voice is recognizable across languages, which is what the audience registers.
What is lost: accent authenticity (a clone speaking a language it was not trained on carries artifacts, and native listeners notice); idiomatic rhythm (sentence stress patterns differ between languages, and a clone tends to apply the source language's rhythm); and emotional register consistency (the drift above is worse in a second language).
The practical approach for the multi-language editions of this book's own subject matter: generate each language from a clone trained on source material in that language where possible, and where not, accept the artifacts and check with a native speaker before publishing. Chapter 6.2's "verify" discipline applies to language the way it applies to model versions.
Directing a synthetic voice
A synthetic voice performs what is written, literally. Paragraph structure, punctuation, and sentence length become the prosody controls, which changes how you write for it.
Punctuation is pacing. A period is a stop; a comma is a breath; an ellipsis is a pause; a dash is an interruption. Writing without punctuation produces a monotone run, and no amount of regeneration fixes text that has no structure to interpret.
Short sentences perform better. A long clause with three subordinate parts gives the model nowhere to place emphasis, and the result is flat. Two short sentences with a pause between them carry the same meaning with audible shape.
Paragraph breaks are real pauses. The model treats a new paragraph as a longer beat. Use them where the speech should pause, not only where the topic changes.
Where the tool supports tags, they are worth using: emphasis, slower rate, whisper, a pause of a specified length. Where it does not, you approximate with punctuation and with a re-generation — a second reading of the same line often lands differently, and picking between takes is cheaper than fighting the model.
Breath is a separate asset. Real speech has audible breaths, and generated speech usually does not. A small library of recorded breath sounds, dropped into the timeline at phrase boundaries (6.7), does more for the impression of a real speaker than any voice-model improvement. It takes ten minutes to record and it is reusable forever.
A practical writing check: read the script aloud before generating it. If it is difficult to read aloud, it will be worse when synthesized. This is the same test as 6.6's script rule, applied at the writing stage rather than after the generation.
Where this goes wrong
Cloning without consent. Chapter 11.4, in full.
A clone built from one emotional register. It will be unable to carry the content you actually plan.
A clone built from noisy source material. Quality problems that no later processing fixes.
Letting the video model speak. Pitch drift within seconds.
Generating video before voice. You have made every audio change a video change.
Skipping the five-line voice test. You will discover the drift in a published post.
Optimizing voice cost. It is the smallest number in the pipeline.
Assuming a clone is accent-neutral across languages. Native listeners hear the artifacts.
The gate
- Is your voice clone (or designed voice) created with documented consent, if cloned at all?
- Was the source material clean, varied, and at least minutes long?
- Have you run the five-line emotional test, and did all five read as the same person?
- Is your pipeline ordered voice → video → lip-sync, with no dialogue generated by a video model?
- Do you know which languages your clone genuinely covers, and have you checked them with a native speaker?
- Is voice cost the smallest line in your budget — and have you stopped optimizing it?
What you leave with
Two routes with the anchor argument for cloning and the consent rule stated at full strength, cost tiers translated into content volume with the conclusion that voice is never the cost problem, four determinants of clone quality, emotional drift with a five-line test and three mitigations, the ordering rule — voice first, never dialogue from a video model — and the three things lost when one clone speaks many languages.