# AI Mind — Process **Role in the pipeline:** the character's conversation system. Retrieval first, generation only inside retrieved sources, with a policy layer enforcing cutoff and refusals. Follows [[Specification — Full-Body Historical Virtual Human]] §3 Stage 5. ## Runtime loop ``` Visitor speech → ASR (speech to text) → intent + safety filter → authored beat match? → play authored clip (no generation) → else retrieval: Corpus A → B → C (hybrid keyword + vector, citation IDs) → LLM answer constrained to retrieved passages, in persona → policy layer: cutoff, refusal list, trauma protocol, companionship ban → streaming TTS (locked actor voice) → audio-driven face (MetaHuman real-time audio / Audio2Face-3D) + gesture tag → captions + log (question, corpus hits, refusal flags; no faces) ``` ## Components (choose per venue) | Component | Options (2026) | Notes | |---|---|---| | ASR | Cloud streaming ASR or on-device model | Museum noise: use a mic array + push-to-talk | | Retrieval store | Postgres + pgvector, or a managed vector DB | One index per figure; never a blended "Nantucket brain" | | LLM | Hosted frontier model API; small open-weights model on the kiosk PC for offline/school venues | Keep model choice behind an interface so it can be swapped | | Policy layer | Rules engine + classifier, separate from the persona prompt | Cutoff is enforced in code, not asked for in the prompt | | TTS | Low-latency streaming TTS with a contracted actor voice (e.g. ElevenLabs class) | Lock the voice; log the version | | Face | MetaHuman real-time audio-driven animation; NVIDIA Audio2Face-3D (open source) | Choose by quality on the target device | | All-in-one character platforms | Convai (Unreal/MetaHuman plugin) and similar | Fast prototypes; check data retention and cutoff control before production | ## Latency budget (target under ~1.5 s to first audio) ASR finalization ~300 ms · retrieval ~100 ms · LLM first tokens ~400–600 ms · TTS first audio ~150–300 ms. Cover gaps with authored "thinking" idles, never filler speech. ## Build steps 1. Corpus ingestion with shelf marks and dated chunks (see [[Mitchell Corpus Inventory]]). 2. Golden question set (400–800) with expected sources and refusal flags. 3. Persona prompt from the voice dossier; policy rules from D-notes. 4. Evaluation harness: run golden set on every corpus or model change; blind historian review of 50 answers. 5. Staff dashboard: pin a canned answer overnight without retraining. 6. Offline fallback: authored monologues only. **Owners:** AI/conversation engineer, conversation designer, historian. Tutorials: [[AI Mind — Tutorials]]