# Tutorial — Speech Out *Augmented Traveler local tutorial.* Back to [[AI Mind — Tutorials]] · the actor and voice bible: [[Voice — Process]] · Unreal playback: [[Tutorial — Conversation Client Plugin]] · speech in: [[Tutorial — Speech In]] How a live answer becomes the figure's voice. We use **one contracted actor voice per figure**, built into a **locked synthetic voice**, streamed as **24 kHz PCM16** to Unreal. It covers consent and contract, the **pronunciation lexicon**, prosody, latency, the **fallback policy** and QA. > **Draft.** Written from the ElevenLabs, Kokoro and Piper docs listed under Sources, and from [[Voice — Process]]. The voice director, the actor and the AI conversation engineer validate it; counsel reviews section 2. Correct setting names here after the first build. ## 1. The rule - **The figure speaks only in its contracted actor's voice.** Never a stock voice, never another actor's. - **If that voice is unavailable,** the figure does not speak live answers. See section 7. - **Credit** the actor on the kiosk and in the out-of-character segment ([[Ground Rules — Historical Virtual Humans]]). ## 2. Consent and the contract Consent is not the same as public domain. The actor signs for the synthetic voice explicitly ([[Voice — Process]] step 2). **Clauses to have in the contract** (counsel drafts; this is a checklist, not legal text): | Clause | What it covers | |---|---| | **Scope** | Synthetic voice for live answers of *this figure only*, on named surfaces (kiosk, headsets, phones, Pixel Streaming) | | **Content limits** | Only answers produced inside our policy layer ([[Tutorial — Policy Layer with Guardrails]]); no ads, no political content, no other characters | | **Retraining** | Whether we may rebuild the voice on a new model or vendor, and whether the actor re-approves | | **Version lock** | The approved voice version is fixed; changes need the voice director's sign-off and the actor's notice | | **Account ownership** | Who holds the cloned voice on the vendor platform (see below) | | **Term and revocation** | How long; what happens at the end (we delete the voice and fall back per section 7) | | **Payment** | Fee, buyout or residuals | | **Credit** | Wording and placement | | **Watermark / disclosure** | The actor agrees the output is labelled synthetic | **ElevenLabs Professional Voice Clone (PVC) — important:** the help centre says you can only create a PVC of **your own** voice, "even with their consent". The actor must create and verify the PVC on **their own account**, then share it with our account through a private sharing link. So: 1. The contract must require the actor to create, verify and share the voice, and not withdraw it during the term. 2. Our account never holds the only copy of the source recordings; the voice bible stays in the studio archive. 3. Check the sharing terms with ElevenLabs before signing (unconfirmed how version changes work on a shared voice; check on first use). ## 3. Streaming TTS set-up **Platform:** ElevenLabs class, as in [[AI Mind — Process]]. What the docs say: | Item | Detail | |---|---| | **Model** | Flash v2.5 is listed at about 75 ms model latency (excluding network and app). Turbo v2.5 is deprecated in favour of Flash. v3 is the most expressive. | | **Voice type and latency** | Default voices and Instant Voice Clones are faster than Professional Voice Clones on Flash v2.5; ElevenLabs says it is working on PVC latency | | **Output format** | `pcm_24000` matches our protocol (24 kHz mono PCM16). `pcm_44100` needs a higher tier | | **Endpoints** | HTTP streaming, or WebSocket `stream-input` for text arriving from the LLM. The WebSocket returns base64 audio with **alignment data** | | **Network** | Time to first byte about 100–150 ms from North America; `api.us.elevenlabs.io` opts out of global routing | | **Determinism** | `seed` (0–4294967295) for repeatable output in QA | | **Continuity** | `previous_text` / `next_text` help prosody across chunks | | **Zero retention** | `enable_logging=false`; enterprise customers only; voice cloning itself is not covered | **Steps:** 1. The service receives LLM text as it streams, already checked by the policy layer. 2. **Cut at sentence or clause ends** before sending to TTS. Half-sentences sound wrong. 3. **Strip anything not to be spoken:** citation markers, stage directions, markup. 4. Send over the WebSocket with the locked settings (section 5). 5. Forward the PCM to Unreal as binary frames. Use the alignment data to time the caption chunks ([[Tutorial — Captions UI]]). **Local alternatives** (offline venues): - **Kokoro-82M:** Apache 2.0, 24 kHz, fixed voices, **no voice cloning**. It cannot be the actor's voice, so it is not a figure voice. - **Piper:** GPL-3.0; can train a new voice; per-voice licences vary. A local voice trained from the actor's recordings is possible in principle, but needs its own contract clause and a quality test (unconfirmed). ## 4. Pronunciation lexicon Owned by the voice director with the historian and dialect coach. One lexicon per figure, versioned. | Word | Note | |---|---| | Siasconset / 'Sconset | Confirm the local pronunciation with the dialect coach | | Wampanoag | Confirm with a Wampanoag advisor, not a dictionary | | Nantucket, Madaket, Polpis, Coatue, Quidnet | Island places | | Atheneum | Institution name | | thee, thy, First Day, Fourth Month | Quaker plain speech (Mitchell, Mott) | | Names of ships and captains (Essex, Veeder) | Whaling | **On ElevenLabs:** 1. Write the lexicon as a **PLS file** (W3C pronunciation lexicon, XML). 2. Use **phoneme** rules (IPA or CMU) where supported. The docs say phoneme tags work only with `eleven_flash_v2` and `eleven_v3`; other models skip them. 3. Use **alias** rules (respelling) where phonemes aren't supported. 4. PLS is **case sensitive**: add "Sconset" and "sconset" both. 5. Up to **3 dictionary locators** per request. Each locator has a dictionary ID and a **version ID**; always pin the version. ## 5. Version lock Log these with every answer and in the venue config: - vendor, `model_id`, `voice_id`; - pronunciation dictionary ID and version ID; - voice settings and seed used in QA; - date approved and who approved it. **Any change** (new model, deprecation, retrained voice) means a new version: repeat section 8 before it goes live. A vendor deprecating a model is a change. ## 6. Prosody - **Short sentences** in the persona prompt ([[Tutorial — Writing a Persona for a Historical Figure]]) give better prosody than long ones. - **Numbers and dates:** write them out in the text ("eighteen forty-seven") so the normaliser can't misread them. - **No filler speech** while waiting; the "thinking" idle covers gaps ([[Tutorial — Conversation Client Plugin]] section 6). - **Refusals** use authored clips where possible, so they always sound right. - **Emotional range:** the voice bible has calm, sharp, grief and humour takes. The gesture tag in `answer_meta` should not fight the voice (a "humour" tag on a flat read looks wrong). ## 7. Fallback policy: no substitute voice | Situation | What happens | |---|---| | TTS vendor down or slow (no audio within the timeout) | Authored line and clip if the question matches one; otherwise the answer shows as **text captions only**, with the figure in a listening idle | | Offline venue | Authored monologues and authored answers only, unless a contracted local voice exists | | Contract ended or revoked | Remove the voice; authored clips only if they are still within their licence | **Never** switch to a stock voice "for now". A different voice is a different person. ## 8. QA before a voice version goes live 1. **Lexicon test:** a script with every lexicon word in two sentences. The dialect coach marks each right or wrong. 2. **50 golden answers** synthesised with the locked settings and seed ([[Tutorial — Evaluating Answers]]). The voice director listens to all 50. 3. **Loudness:** match the authored monologues (the sound designer sets the target). 4. **Latency:** time to first audio from the service, 50 runs, on the venue network. 5. **Watermark check:** ElevenLabs says it embeds an inaudible watermark in its audio and offers an Audio Detector. Run one sample through it and file the result. 6. **Actor review:** send the actor a sample reel before first opening. ## 9. Practice exercise Take one Mitchell answer about the 1847 comet. Synthesise it three ways: as one block, cut at sentences, and cut at sentences with `previous_text`. Then add "Siasconset" and "Wampanoag" to a test lexicon and synthesise again. Note which version the voice director prefers, and why. ## 10. Check before a venue build - [ ] Actor contract signed with the clauses in section 2; voice shared correctly - [ ] Voice version, model, dictionary version and seed recorded - [ ] `pcm_24000` reaches Unreal; lip sync follows it - [ ] Lexicon test passed; dialect coach sign-off - [ ] Fallback tested: vendor blocked → captions and authored clips, no stock voice - [ ] Zero-retention setting confirmed with the vendor contract, or the gap written down ([[Tutorial — Privacy and Logging]]) ## Sources - ElevenLabs, Stream speech API: https://elevenlabs.io/docs/api-reference/text-to-speech/stream - ElevenLabs, WebSocket stream-input: https://elevenlabs.io/docs/api-reference/text-to-speech/v-1-text-to-speech-voice-id-stream-input - ElevenLabs, Models: https://elevenlabs.io/docs/overview/models - ElevenLabs, Latency optimisation: https://elevenlabs.io/docs/eleven-api/guides/how-to/best-practices/latency-optimization - ElevenLabs, Using pronunciation dictionaries: https://elevenlabs.io/docs/eleven-api/guides/how-to/text-to-speech/pronunciation-dictionaries - ElevenLabs, Can I create a Professional Voice Clone of someone else's voice?: https://elevenlabs.io/docs/help-center/product/voice-customization/voice-cloning/can-i-create-a-professional-voice-clone-of-someone-elses-voice - ElevenLabs, Audio detector: https://elevenlabs.io/docs/eleven-creative/audio-tools/audio-detector - ElevenLabs, Zero Retention Mode (Enterprise): https://elevenlabs.io/docs/eleven-api/resources/zero-retention-mode - hexgrad, Kokoro-82M: https://huggingface.co/hexgrad/Kokoro-82M - OHF-Voice, Piper: https://github.com/OHF-Voice/piper1-gpl - Full list: [[Sources — 2026-09-20]]