# Historical Persona AI — Glossary and Sources Companion note to the conversation on building a **text or speech historical-character interface** grounded in a real person's writings and contemporary knowledge. > [!info] How to use this in Obsidian > - Terms are written so you can `[[wikilink]]` them later (e.g. `[[RAG]]`). > - Sources are grouped by topic. URLs are plain markdown links. > - Callouts mark definitions vs. caveats. --- ## Glossary ### Core methods #### RAG **Retrieval-Augmented Generation.** Instead of asking the model to answer from memory alone, the system first *searches a document collection you control*, then feeds the retrieved passages into the prompt so the answer is grounded in those texts. For a historical character this is the main accuracy tool: letters, speeches, and dated notes are retrieved before the model speaks. It does **not** by itself stop anachronism; you still need date filters and refusal rules. Related: [[vector database]], [[embedding]], [[hybrid search]], [[BM25]], [[lorebook]]. #### LoRA **Low-Rank Adaptation.** A way to *fine-tune* a large model by training a small number of extra weights instead of rewriting the whole model. The result is a compact “adapter” you can load on top of the base model. Use LoRA for *voice and rhetorical habit* (sentence rhythm, favorite metaphors). Do **not** expect LoRA alone to erase the model’s modern world knowledge. Related: [[QLoRA]], [[PEFT]], [[fine-tuning]], [[SFT]]. #### QLoRA **Quantized LoRA.** LoRA plus **quantization** (storing the frozen base model in 4-bit, typically). Same idea as LoRA, much lower VRAM. The usual method if you fine-tune an 8B–32B model on a single consumer GPU. #### OCR **Optical Character Recognition.** Software that turns photographed or scanned pages (PDF page images, newspaper scans, manuscript photos) into searchable text. Primary-source pipelines almost always start here. Dirty OCR (long-s as `f`, broken line wraps, running headers) will pollute retrieval, so clean it before chunking. #### LLM **Large Language Model.** The generative engine (Claude, GPT-class, Gemini, Llama, Qwen, GLM, Kimi, etc.) that writes the character’s reply. #### PEFT **Parameter-Efficient Fine-Tuning.** Family of methods that adapt a model without full retraining. [[LoRA]] and [[QLoRA]] are the common members. #### SFT **Supervised Fine-Tuning.** Training on example pairs: *user question → in-character answer*. Better for a persona than dumping raw collected works into next-token pretraining, especially if you include “I would not know that” examples. #### DPO **Direct Preference Optimization.** A fine-tuning method that teaches the model to prefer one reply over another (e.g. historian-rated authentic vs. anachronistic). Used in some character-roleplay papers together with SFT. #### RLHF **Reinforcement Learning from Human Feedback.** Older / heavier cousin of DPO: a reward model scores answers, then the policy is optimized. Mentioned in some historical-chatbot writeups as a way to reward “sounds like Churchill in 1943” over “explains the internet.” --- ### Memory, search, and prompting #### embedding A numeric vector that represents the *meaning* of a text chunk. Similar passages land near each other, which is how semantic search finds “that letter about the harvest” even if the user did not use those words. #### vector database Store for embeddings plus metadata (date, place, source type). Common tools: **Chroma**, **Qdrant**, **LanceDB**. #### hybrid search Combine semantic search (embeddings) with lexical search ([[BM25]]). Useful when the user quotes a proper name, a statute title, or a distinctive 19th-century phrase. #### BM25 **Best Matching 25.** A classic keyword ranking function. Complements embeddings; does not replace them. #### chunk / chunking Splitting long documents into retrieval-sized pieces (a letter, a speech paragraph, a diary entry). For historical work, *document-and-date-aware* chunks beat naive “every 500 tokens.” #### metadata Fields attached to each chunk: `source_date`, `place`, `source_type` (letter / speech / diary / later biography), `certainty`, `first_person`. Date metadata is what lets you refuse posthumous knowledge. #### lorebook Also called **World Info** in [[SillyTavern]]. Keyword-triggered snippets inserted into the prompt only when those keys appear. Lighter than full RAG; good for stable facts (birth year, offices held, family names). #### character card A portable persona file (JSON or PNG-with-metadata, Character Card V2/V3). Holds description, personality, scenario, first message, example dialogue, and sometimes an embedded lorebook. #### few-shot Including a handful of example Q&A turns in the prompt so the model copies register and refusal style. #### system prompt The hidden instruction that defines identity, “today’s” date, ignorance rules, and citation habits. #### guardrail A second pass (rules or a small model) that flags anachronisms, modern brand names, post-death dates, or later scientific explanations, then forces a regenerate or a refusal. #### hallucination Fluent invention presented as fact. RAG reduces it; it does not eliminate it. Always keep a “show the retrieved passage” toggle while you are building. #### knowledge cutoff / temporal bound The last date the *character* is allowed to know — usually death date or a chosen “now” (e.g. 1863 Lincoln). Separate from the *model’s* training cutoff. --- ### Training and hardware #### fine-tuning Any extra training after the base model exists. Full fine-tune = change all weights (expensive). [[LoRA]] / [[QLoRA]] = change a thin adapter. #### pretraining The original internet-scale training run. You will not redo this for one historical figure. #### LoRA rank (`r`) Size of the adapter. Common starting point: `r = 16`. Higher rank = more capacity and more risk of overfitting a small letter corpus. #### DoRA **Weight-Decomposed Low-Rank Adaptation.** A LoRA variant some 2026 toolchains default to. #### Unsloth Single-GPU fine-tuning toolkit; fast LoRA/QLoRA on consumer cards. #### Axolotl YAML-configured fine-tuning pipeline; better when you want reproducible multi-method runs (SFT, DPO, etc.). #### LLaMA-Factory Fine-tuning UI / toolkit covering many alignment methods. #### VRAM **Video RAM.** GPU memory. Determines whether a local model or a QLoRA job fits. #### GPU / CPU / CUDA / MPS **Graphics Processing Unit** / **Central Processing Unit** / NVIDIA’s compute platform / Apple Silicon GPU backend. Local inference and training care which you have. #### quantization (4-bit, Q4, etc.) Storing weights at lower precision so a bigger model fits in less VRAM, with some quality cost. --- ### Speech interface #### STT **Speech-to-Text** (also ASR, **Automatic Speech Recognition**). Microphone → transcript. Examples: Whisper, Parakeet, Deepgram, AssemblyAI. #### TTS **Text-to-Speech.** Model reply → audio. Examples: ElevenLabs, Cartesia, Kokoro, Chatterbox, GPT-SoVITS, Fish Audio. #### TTFA **Time To First Audio.** Latency until speech starts. Matters for spoken conversation, not for a typed prototype. #### WER **Word Error Rate.** STT accuracy metric. Lower is better. #### MOS **Mean Opinion Score.** Human listening-quality rating, usually 1–5. Used in TTS papers. #### voice cloning Synthesizing speech that matches a reference recording. Legally and ethically the sharp edge for real historical figures; a *period-plausible* voice is usually safer than cloning a specific performance. #### realtime / speech-to-speech Pipelines that skip a separate text step (e.g. OpenAI Realtime). Convenient, harder to inspect for anachronism because you never see the intermediate prose. --- ### Legal and identity (from the source pass) #### NIL **Name, Image, and Likeness.** The commercial identity that publicity-rights law protects. #### right of publicity State-level (in the U.S.) right to control commercial use of identity. Often **post-mortem** for a fixed term (examples discussed in sources: California commonly cited at 70 years, Texas 50, New York shorter / performer-focused). Not uniform nationally. #### digital replica Statutory term in newer AI bills and California A.B. 1836: a computer-generated, highly realistic voice or visual likeness. Commercial audiovisual use of a deceased personality’s replica may need estate consent; commentary, criticism, parody, scholarship, and some documentary/biographical uses are often carved out — **check counsel**, do not treat this note as legal advice. #### public domain Works whose copyright has expired (in the U.S., generally author death + 70 years for many works, with older rules for pre-1978 publications). **Their letters** may be free to index; **a 2019 annotated biography** usually is not. --- ### Other acronyms that showed up in the research | Acronym | Expansion | Why it appeared | |---|---|---| | API | Application Programming Interface | How hosted models and TTS/STT are called | | UI | User Interface | Chat frontends | | PDF | Portable Document Format | Source scans | | KG | Knowledge Graph | Some biography systems (e.g. AIstorian) index facts as graphs, not only passages | | MEI | Major Entity Identification | Entity/coreference step in Fic2Bot | | GDPR | General Data Protection Regulation | EU living-person data law; generally does not apply to the long-dead, but *user* chat logs still might | | HIPAA | Health Insurance Portability and Accountability Act | Irrelevant unless you store health data; appeared only in voice-platform comparison tables | | PNG | Portable Network Graphics | SillyTavern character cards often embed JSON in a PNG avatar | --- ## Suggested Obsidian map ``` [[Historical Persona AI — Glossary and Sources]] ├── [[RAG]] ├── [[LoRA]] / [[QLoRA]] ├── [[OCR]] ├── [[character card]] ├── [[lorebook]] └── Sources ├── Methods (RAG, episodic memory, role-play) ├── Tooling (frontends, fine-tune, voice) ├── Primary-text repositories └── Law and ethics ``` --- ## Sources Collected from the research pass on 2026-09-19. Annotations say *why* the item matters for this project, not that every claim in the paper was independently re-verified. ### Methods — RAG, persona, historical dialogue - Arias Gonzalez, Rafael, and Steve DiPaola. “Cognitively-Inspired Episodic Memory Architectures for Accurate and Efficient Character AI.” arXiv, 16 Nov. 2025. [PDF](https://www.arxiv.org/pdf/2511.10652). Van Gogh case study: biographical data turned into 1,774 dated first-person “memories” with affective metadata; two-stage retrieval ~0.52s. Best single paper for the “lived memory, not encyclopedia” pattern. - Digital Orientalist. “Voices from the Past: Retrieval-Augmented Dialogues with Chinese Historical Figures.” 26 Dec. 2025. [https://digitalorientalist.com/2025/12/26/voices-from-the-past-retrieval-augmented-dialogues-with-chinese-historical-figures/](https://digitalorientalist.com/2025/12/26/voices-from-the-past-retrieval-augmented-dialogues-with-chinese-historical-figures/). Classroom RAG personas (Li Hongzhang, Lu Xun, Sun Yat-sen, and others). Stresses showing retrieved sources to students. - Kurz, M. *histfig*. GitHub. [https://github.com/mcjkurz/histfig](https://github.com/mcjkurz/histfig). Runnable FastAPI app: upload documents, hybrid RAG, historical/fictional figure chat. Companion to the Digital Orientalist writeup. - Chang, Michael. “Data-Driven Characters.” LangChain blog, 19 June 2023. [https://www.langchain.com/blog/data-driven-characters](https://www.langchain.com/blog/data-driven-characters). Early “upload a corpus, name a character” recipe; still a clear mental model. - Park, Jeiyoon, et al. “Dynamic Context Adaptation for Consistent Role-Playing Agents with Retrieval-Augmented Generations.” arXiv:2508.02016. [https://arxiv.org/html/2508.02016v4](https://arxiv.org/html/2508.02016v4). Training-free Amadeus framework + CharacterRAG dataset; persona consistency when the question is *outside* the character’s knowledge. - “Fic2Bot: A Scalable Framework for Persona-Driven Chatbot Generation from Fiction.” *Electronics* 14.19 (2025). [https://www.mdpi.com/2079-9292/14/19/3859/htm](https://www.mdpi.com/2079-9292/14/19/3859/htm). Scene-level RAG + stylistic profiling from raw narrative text (fiction-oriented, transferable chunking ideas). - Han, Xiaoxu, et al. “Act-LLM: A whole-process chain for character-centric role-playing with large language models.” *Expert Systems with Applications*, 2026. [ScienceDirect](https://www.sciencedirect.com/science/article/pii/S0957417425026417). PEFT + DPO aimed at **personality and knowledge boundaries** together — the paper closest to “limited to what this person could know.” - “Beyond Profile: From Surface-Level Facts to Deep Persona Simulation in LLMs.” arXiv:2502.12988. [https://arxiv.org/html/2502.12988](https://arxiv.org/html/2502.12988). CharacterBot / Lu Xun: style and thought-pattern tasks from 17 essay collections. Includes an ethics paragraph on misrepresentation. - “A Platform for Interactive AI Character Experiences” (Digital Einstein). arXiv:2601.01027. [https://arxiv.org/html/2601.01027](https://arxiv.org/html/2601.01027). GPT-4o + fine-tuned Llama 3, synthetic conversations, embedding steering, museum-style install. - “A Preliminary Study of RAG for Taiwanese Historical Archives.” arXiv:2511.07445. [https://arxiv.org/html/2511.07445](https://arxiv.org/html/2511.07445). Historical Traditional-Chinese corpora; metadata and query design effects on retrieval. - “AIstorian lets AI be a historian: A KG-powered multi-agent system for accurate biography generation.” arXiv:2503.11346. [https://arxiv.org/html/2503.11346](https://arxiv.org/html/2503.11346). Knowledge-graph RAG + anti-hallucination agents + style fine-tune. Biography generation, not live chat, but the KG index idea travels. - Estha. “How to Create Historical Persona Bots for Education: A Complete Guide.” 6 Feb. 2026. [https://estha.ai/blog/how-to-create-historical-persona-bots-for-education-a-complete-guide/](https://estha.ai/blog/how-to-create-historical-persona-bots-for-education-a-complete-guide/). Practitioner checklist: identity, voice, knowledge base, primary sources first. - Next Electronics. “AI Chatbot to Simulate Historical Characters.” [https://www.next.gr/ai/text-generation/ai-chatbot-to-simulate-historical-characters](https://www.next.gr/ai/text-generation/ai-chatbot-to-simulate-historical-characters). Survey-style notes on primary vs secondary sources, hybrid RAG, and a Churchill fine-tune anecdote (treat numbers as illustrative, not gospel). ### Tooling — frontends, roleplay models, fine-tuning - AIREITER. “Best AI Model for Roleplay: Consistency, Memory, and API Access.” Updated 15 Sep. 2026. [https://aireiter.com/blog/best-ai-model-for-roleplay](https://aireiter.com/blog/best-ai-model-for-roleplay). Snapshot of which hosted/open models people were picking for *character consistency* vs *canon* vs local control in mid-September 2026. Rankings age fast. - Aituts. “SillyTavern Guide 2026: Setup, Character Cards, Presets & Lorebooks.” 18 Sep. 2026. [https://aituts.com/sillytavern-guide/](https://aituts.com/sillytavern-guide/). - SillyTavern Docs. “Data Bank.” [https://github.com/SillyTavern/SillyTavern-Docs/blob/main/Usage/Characters/data-bank.md](https://github.com/SillyTavern/SillyTavern-Docs/blob/main/Usage/Characters/data-bank.md). Built-in RAG attachments + local embeddings. - LumiChat. “SillyTavern Lorebook Guide (2026): World Info & Triggers.” [https://www.lumichat.ink/blog/sillytavern-lorebook-world-info-guide](https://www.lumichat.ink/blog/sillytavern-lorebook-world-info-guide). Character lore vs chat lore vs global World Info — useful when you do **not** want later-historian notes leaking into every turn. - CodersEra. “Fine-Tuning LLMs in 2026: LoRA, QLoRA, Unsloth, MLX.” 27 May 2026. [https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/](https://codersera.com/blog/fine-tuning-llms-complete-guide-2026/). - Misar. “Unsloth vs Axolotl: Which Is Better in 2026?” [https://www.misar.blog/compare/unsloth-vs-axolotl-fine-tuning](https://www.misar.blog/compare/unsloth-vs-axolotl-fine-tuning). - local-llm.net. “Best Local LLM Fine-Tuning Tools 2026.” [https://www.local-llm.net/best/fine-tuning/](https://www.local-llm.net/best/fine-tuning/). - Unsloth Docs. “How to Fine-tune LLMs with Unsloth & Docker.” [https://unsloth.ai/docs/blog/how-to-fine-tune-llms-with-unsloth-and-docker](https://unsloth.ai/docs/blog/how-to-fine-tune-llms-with-unsloth-and-docker). ### Voice (STT / TTS) Use these only after the text persona is stable. - The AI Bench. “Local voice AI stack 2026 — TTS, cloning, STT.” 19 Apr. 2026. [https://theaibench.ai/guides/voice-tts-local-stack/](https://theaibench.ai/guides/voice-tts-local-stack/). Kokoro, MOSS-TTS-Nano, Chatterbox, Parakeet; Open-WebUI as the glue layer. - Inworld. “Voice AI for AI Character Apps.” [https://inworld.ai/resources/voice-ai-for-ai-character-apps](https://inworld.ai/resources/voice-ai-for-ai-character-apps). - FutureAGI. “Best Voice AI April 2026: Native Audio, STT, TTS.” [https://futureagi.com/blog/best-voice-ai-april-2026/](https://futureagi.com/blog/best-voice-ai-april-2026/). - The AI Rankings. “Best AI Voice Clone in 2026.” June 2026. [https://theairankings.com/best-ai-voice-clone/](https://theairankings.com/best-ai-voice-clone/). - Hume AI. “Introducing the Hume Voice Replication Leaderboard.” 10 Sep. 2026. [https://www.hume.ai/blog/introducing-the-hume-voice-replication-leaderboard](https://www.hume.ai/blog/introducing-the-hume-voice-replication-leaderboard). Reminder: *natural* ≠ *sounds like that specific person*. - Bland AI. “11 Best TTS for AI Voice Agents Ranked & Tested 2026.” [https://www.bland.ai/blog/best-tts-for-ai-voice-agents](https://www.bland.ai/blog/best-tts-for-ai-voice-agents). - ai4s-research. *awesome-text-to-speech*. GitHub. [https://github.com/ai4s-research/awesome-text-to-speech](https://github.com/ai4s-research/awesome-text-to-speech). ### Primary-text repositories (examples, not exhaustive) - Project Gutenberg. Example: *Speeches & Letters of Abraham Lincoln, 1832–1865*. [https://www.gutenberg.org/ebooks/14721](https://www.gutenberg.org/ebooks/14721). - Internet Archive mirror of the same: [https://archive.org/details/speechesandlette14721gut](https://archive.org/details/speechesandlette14721gut). - National Archives / Oxford. *Our Documents: 100 Milestone Documents*. [https://archive.org/details/ourdocuments100m0000unit](https://archive.org/details/ourdocuments100m0000unit). - Online Library of Liberty / similar collected-charter anthologies — example contents list: [https://oll-resources.s3.us-east-2.amazonaws.com/oll3/store/titles/669/Frohnen_0082.html](https://oll-resources.s3.us-east-2.amazonaws.com/oll3/store/titles/669/Frohnen_0082.html). Also search, once you pick a person: Founders Online, presidential libraries, collected-papers projects, Chronicling America, HathiTrust, national archives of the relevant country. ### Law and ethics (orientation only — not legal advice) - Wake Forest Journal of Law & Policy. “Deepfakes of the Dead: Applying Postmortem Publicity Law to Artificial Intelligence Digital Replicas.” [https://wfujournaloflawandpolicy.org/deepfakes-of-the-dead-applying-postmortem-publicity-law-to-artificial-intelligence-digital-replicas/](https://wfujournaloflawandpolicy.org/deepfakes-of-the-dead-applying-postmortem-publicity-law-to-artificial-intelligence-digital-replicas/). - WCR.LEGAL. “Post-Mortem Digital Identity: Legal Framework 2026.” [https://wcr.legal/post-mortem-digital-identity-legal-framework-2026/](https://wcr.legal/post-mortem-digital-identity-legal-framework-2026/). Useful comparison table mindset: publicity rights vs copyright vs data-protection, and that domicile-at-death often matters more than where the app is incorporated. - Expert Institute. “Postmortem Publicity Rights and Digital Use of Celebrity Likeness.” [https://www.expertinstitute.com/resources/insights/ai-digital-celebrity-likeness-right/](https://www.expertinstitute.com/resources/insights/ai-digital-celebrity-likeness-right/). - POLITICO Digital Future Daily. “The tricky business of AI cloning.” 9 July 2026. [https://www.politico.com/newsletters/digital-future-daily/2026/07/09/the-tricky-business-of-ai-cloning-00992279](https://www.politico.com/newsletters/digital-future-daily/2026/07/09/the-tricky-business-of-ai-cloning-00992279). - CDAS. “California Expands Its Post-Mortem Right of Publicity Law to Cover AI Digital Replicas.” [https://cdas.com/california-expands-its-post-mortem-right-of-publicity-law-to-cover-ai-digital-replicas/](https://cdas.com/california-expands-its-post-mortem-right-of-publicity-law-to-cover-ai-digital-replicas/). A.B. 1836 / Cal. Civ. Code § 3344.1 and the scholarship / commentary / documentary exceptions as described by practitioners. - ResearchGate / conference paper. “Responsibility Toward Truth — Ethical Standards in the Development of AI Simulations of Historical Figures.” [https://www.researchgate.net/publication/399330364_RESPONSIBILITY_TOWARD_TRUTH_-ETHICAL_STANDARDS_IN_THE_DEVELOPMENT_OF_AI_SIMULATIONS_OF_HISTORICAL_FIGURES_ODGOVORNOST_PREMA_ISTINI_-ETICKI_STANDARDS_U_RAZVOJU_AI_SIMULACIJA_ISTORIJSKIH_LICNOSTI](https://www.researchgate.net/publication/399330364_RESPONSIBILITY_TOWARD_TRUTH_-ETHICAL_STANDARDS_IN_THE_DEVELOPMENT_OF_AI_SIMULATIONS_OF_HISTORICAL_FIGURES_ODGOVORNOST_PREMA_ISTINI_-ETICKI_STANDARDS_U_RAZVOJU_AI_SIMULACIJA_ISTORIJSKIH_LICNOSTI). > [!warning] Label the reconstruction > Several of the persona papers argue that users should be told this is a text-grounded simulation, not the person. That is both an ethics practice and a practical defense when the model is wrong. --- ## One-page stack reminder 1. **Closed corpus** of dated primary texts (after [[OCR]] cleanup). 2. **[[RAG]]** with date metadata — no live web in character mode. 3. **System prompt + [[guardrail]]** for ignorance and anachronism. 4. Optional **[[LoRA]] / [[QLoRA]]** on 500–2,000 grounded Q&A pairs for voice. 5. **[[STT]] / [[TTS]]** only after the prose persona holds up under red-team questions. --- ## See also - Previous conversation plan: pick one figure with a large public-domain paper trail → timeline + voice sheet + ignorance sheet → SillyTavern or histfig + RAG → red-team modern questions → then voice → then LoRA.