quot;Unknown command '{parts[0]}'. Try 'help'.", "term__line--error"); } void PrintHelp() { Enqueue("help list commands"); Enqueue("status show systems"); Enqueue("doors open|close"); Enqueue("power on|off"); Enqueue("clear clear the screen"); } void PrintStatus() { Enqueue(quot;POWER ..... {(powerOn ? "ONLINE" : "OFFLINE")}"); Enqueue(quot;DOORS ..... {(doorsOpen ? "OPEN" : "SEALED")}"); } void Toggle(string[] args, string label, Action<bool> apply) { if (args.Length != 1 || (args[0] != "on" && args[0] != "off" && args[0] != "open" && args[0] != "close")) { Enqueue(quot;usage: {label} <on|off|open|close>", "term__line--error"); return; } bool value = args[0] == "on" || args[0] == "open"; apply(value); Enqueue(quot;{label.ToUpperInvariant()} -> {args[0].ToUpperInvariant()}"); } // ---- Typewriter --------------------------------------------------- void Enqueue(string text, string cssClass = null) => pending.Enqueue((text, cssClass)); void Pump() { if (currentLine == null) { if (pending.Count == 0) return; var (text, cssClass) = pending.Dequeue(); currentText = text; shown = 0; currentLine = new Label(string.Empty); currentLine.AddToClassList("term__line"); if (cssClass != null) currentLine.AddToClassList(cssClass); output.Add(currentLine); } shown = Mathf.Min(currentText.Length, shown + charsPerTick); currentLine.text = currentText.Substring(0, shown); output.ScrollTo(currentLine); if (shown >= currentText.Length) currentLine = null; } static T Require<T>(VisualElement root, string name) where T : VisualElement { var e = root.Q<T>(name); if (e == null) throw new InvalidOperationException(quot;UXML is missing {typeof(T).Name} named '{name}'"); return e; } } ``` What to take from this: - **Trickle-down + `StopImmediatePropagation`**: the `TextField` handles Enter itself (and Up/Down move its caret). Trickle-down callbacks on the field run before its own default action, so you can claim the key. - **`Focus()` on the next tick** (`schedule.Execute`): focusing in the same call that changed state (or before the panel has a frame) often does nothing. - **Queue + single pump**: `help` enqueues five lines; they type sequentially. A scheduled item per line would type them all at once. - **`Substring` per tick** allocates a little garbage; for terminals this short it is irrelevant. For a long log use `TextElement` with a `StringBuilder` and set `text` once per tick. - The command table is the extension point. The terminal knows no game objects; it raises `UnityEvent<bool>`s and mirrors state in via `SetDoorsOpen` / `SetPowerOn`. ## Step A5 - The screen mesh 1. Create a **Quad** (`GameObject > 3D Object > Quad`) named `TerminalScreen`, scaled `(1.6, 1.0, 1)`, placed on the wall prop. The Quad primitive ships with a `MeshCollider`; keep it. `RaycastHit.textureCoord` is only valid for `MeshCollider` hits (a box or sphere collider returns zero). 2. Create a material (URP/Unlit is the easiest): **Base Map = `TerminalRT`**. For a glowing look use URP/Lit with Base Map black-ish, **Emission enabled, Emission Map = `TerminalRT`**, and a Global Volume with Bloom. 3. Put the quad on a layer `TerminalScreen` so the raycast can ignore everything else. 4. If the image looks mirrored or upside down, the Quad UVs are standard, so look at the two Y flips in A6 first. ## Step A6 - `TerminalUseMode.cs` (pointer mapping + use mode) This combines the official UV mapping sample with the "use mode" interaction model. ```csharp using UnityEngine; using UnityEngine.Events; using UnityEngine.InputSystem; using UnityEngine.UIElements; public class TerminalUseMode : MonoBehaviour { [Header("Wiring")] [SerializeField] PanelSettings panelSettings; // TerminalPanelSettings [SerializeField] TerminalController terminal; [SerializeField] Collider screenCollider; // the quad's MeshCollider [SerializeField] Camera playerCamera; // defaults to Camera.main [SerializeField] LayerMask screenLayers = ~0; [Header("Interaction")] [SerializeField, Min(0.5f)] float useDistance = 2.2f; [SerializeField] UnityEvent<bool> onUseModeChanged; // disable look/move scripts here [SerializeField] GameObject hintObject; // optional "[E] Use" world hint public bool InUse { get; private set; } void Awake() { if (playerCamera == null) playerCamera = Camera.main; } void OnEnable() => panelSettings.SetScreenToPanelSpaceFunction(ScreenToPanel); void OnDisable() { panelSettings.SetScreenToPanelSpaceFunction(null); if (InUse) SetUse(false); } void Update() { var kb = Keyboard.current; if (kb == null) return; if (InUse) { if (kb.escapeKey.wasPressedThisFrame) SetUse(false); return; } bool looking = IsLookingAtScreen(); if (hintObject != null) hintObject.SetActive(looking); if (looking && kb.eKey.wasPressedThisFrame) SetUse(true); } bool IsLookingAtScreen() { var ray = new Ray(playerCamera.transform.position, playerCamera.transform.forward); return Physics.Raycast(ray, out var hit, useDistance, screenLayers, QueryTriggerInteraction.Ignore) && hit.collider == screenCollider; } void SetUse(bool use) { InUse = use; Cursor.lockState = use ? CursorLockMode.None : CursorLockMode.Locked; Cursor.visible = use; if (hintObject != null && use) hintObject.SetActive(false); terminal.SetActive(use); onUseModeChanged?.Invoke(use); } // Called by UI Toolkit for every pointer event. Input is in UI Toolkit screen // coordinates (origin top-left). Return panel pixels, or NaN to say "not over this panel". Vector2 ScreenToPanel(Vector2 screenPosition) { var miss = new Vector2(float.NaN, float.NaN); if (!InUse) return miss; // ignore the mouse unless we are using it screenPosition.y = Screen.height - screenPosition.y; // to Camera screen space (origin bottom-left) var ray = playerCamera.ScreenPointToRay(screenPosition); if (!Physics.Raycast(ray, out var hit, 20f, screenLayers, QueryTriggerInteraction.Ignore)) return miss; if (hit.collider != screenCollider) return miss; var rt = panelSettings.targetTexture; var uv = hit.textureCoord; // needs a MeshCollider return new Vector2(uv.x * rt.width, (1f - uv.y) * rt.height); // flip Y back to top-left origin } } ``` Details to understand: - **Why two Y flips**: UI Toolkit hands you coordinates with a top-left origin; `Camera.ScreenPointToRay` expects bottom-left; the hit UV is bottom-left; the panel wants top-left. The official sample does exactly this. - **Returning NaN** tells the panel the pointer is not over it, so hover states clear and clicks are ignored. We also return NaN when not in use mode so the terminal ignores the (locked, center-screen) cursor while you walk around. - `SetScreenToPanelSpaceFunction(null)` restores default behaviour (as in the sample's `OnDisable`). - If the player camera uses a **Render Scale / dynamic resolution / viewport rect** that differs from `Screen`, adapt the Y flip and viewport mapping. - The raycast must hit the same collider the player sees. Disable other colliders in front (props, trigger volumes) or restrict with `screenLayers`. - For crosshair/gamepad mode, replace the first lines with a ray from the camera center and drive clicks from a button action; see [[5.Diegetic UI Approaches]]. ## Step A7 - Scene wiring 1. `Terminal` GameObject: **Panel Renderer** with `TerminalPanelSettings` + `Terminal.uxml`, plus `TerminalController`. It is not drawn on the screen; it renders to the texture. Keep it enabled. 2. `TerminalScreen` quad: material from A5, `MeshCollider`. 3. `TerminalUseMode` (can sit on the same GameObject as the quad): assign Panel Settings, Terminal, Screen Collider, camera. In `On Use Mode Changed`, call the `enabled` of your look and move scripts with the bool **inverted** (a tiny adapter such as `public void SetGameplayInput(bool useMode) => look.enabled = move.enabled = !useMode;`). 4. Wire `onDoorsRequested` to your door script's `SetOpen(bool)`. 5. Play: walk up, press **E**, click the field, type `help`, press **Enter**, use the buttons, press **Esc**. If it does not work: [[7.UI Toolkit Pitfalls and Performance]] section 7.4 lists diagnostics (texture black, clicks offset, keys moving the player). --- # Part B - The wrist display A flat readout parented to the hand. It uses a **World Space** panel, which is the right tool for non-interactive, always-near-the-camera UI. ## Step B1 - Panel Settings `Assets/UI/Settings/WristPanelSettings.asset`: | Property | Value | Why | |---|---|---| | Render Mode | **World Space** | Panel is a scene object | | Pixels Per Unit | **4000** | UI pixels per world unit (meter). Author the UI 400 x 240 px => 0.10 m x 0.06 m on the wrist | | Scale Mode | Constant Pixel Size | Sizing is physical here, not screen-relative | | Theme Style Sheet | default runtime theme | | Pixels Per Unit lives in **PanelSettings**, so a world-space asset cannot be shared with the screen HUD. ## Step B2 - UXML and USS `Assets/UI/Wrist/Wrist.uxml` ```xml <ui:UXML xmlns:ui="UnityEngine.UIElements" xmlns:game="Game.UI" editor-extension-mode="False"> <Style src="project://database/Assets/UI/Shared/Theme.uss" /> <Style src="project://database/Assets/UI/Hud/Hud.uss" /> <Style src="Wrist.uss" /> <ui:VisualElement name="wrist-root" class="theme wrist"> <ui:VisualElement class="wrist__row"> <ui:Label text="HDG" class="caption" /> <ui:Label name="wrist-heading" class="wrist__big" /> </ui:VisualElement> <game:ResourceBar name="wrist-hp" label="HP" /> <ui:VisualElement class="wrist__row"> <ui:Label text="T+" class="caption" /> <ui:Label name="wrist-clock" class="wrist__big" /> </ui:VisualElement> </ui:VisualElement> </ui:UXML> ``` `Assets/UI/Wrist/Wrist.uss` ```css .wrist { width: 400px; height: 240px; padding: 20px; justify-content: space-between; background-color: rgba(8, 20, 30, 0.85); border-width: 3px; border-color: var(--color-accent); border-radius: 16px; } .wrist__row { flex-direction: row; justify-content: space-between; align-items: center; } .wrist__big { font-size: 40px; -unity-font-style: bold; color: var(--color-accent); } /* Bigger bar for a screen that is small in the world */ .wrist .resource-bar { height: 44px; } .wrist .resource-bar__label { font-size: 26px; } ``` Type is large relative to the 400 px canvas because the whole panel is 10 cm wide. Judge sizes by how the wrist looks in the headset/camera, not on the UI Builder canvas. ## Step B3 - Place the panel 1. In the player rig, add an empty `WristAnchor` as a child of the hand/forearm bone, positioned and rotated on the inside of the wrist. 2. Child `WristDisplay`: **Panel Renderer** with `WristPanelSettings` and `Wrist.uxml`. 3. In the component's **World-Space Dimensions** section set the size so the panel is **0.10 x 0.06** world units (Fixed size mode with those values, matching 400 x 240 px at 4000 Pixels Per Unit; if your Unity version labels the modes differently, the goal is that panel width in meters = pixel width / Pixels Per Unit) and pivot **Center**. Use the Scene view to check it against the hand model. 4. Which side is the front? World-space panels are single-sided (unlit quad). If you only see it from behind or it is mirrored, rotate `WristDisplay` 180 degrees around Y (and see `flipFacing` below). 5. Optional interaction: add a `BoxCollider` (0.10 x 0.06 x 0.005) on a dedicated layer and a **Panel Input Configuration** (`GameObject > UI Toolkit > Panel Input Configuration`) with that layer in *Interaction Layers* and your camera in *Event Cameras*. Read-only readouts do not need either. ## Step B4 - `WristDisplay.cs` ```csharp using System; using Unity.Properties; using UnityEngine; using UnityEngine.UIElements; [RequireComponent(typeof(PanelRenderer))] public class WristDisplay : MonoBehaviour { [SerializeField] PlayerStats stats; [SerializeField] Camera viewCamera; [Header("Raise-to-view")] [SerializeField] bool flipFacing; // tick if your panel faces the other way [SerializeField, Range(0f, 1f)] float showAbove = 0.55f; // facing the camera more than this => show [SerializeField, Range(0f, 1f)] float hideBelow = 0.40f; // facing less than this => hide (hysteresis) [SerializeField, Min(0f)] float maxDistance = 0.9f; // meters PanelRenderer panelRenderer; int loadedVersion = -1; Label headingLabel, clockLabel; bool visible; void Awake() { panelRenderer = GetComponent<PanelRenderer>(); if (viewCamera == null) viewCamera = Camera.main; } void OnEnable() => panelRenderer.RegisterUIReloadCallback(OnUIReload); void OnDisable() => panelRenderer.UnregisterUIReloadCallback(OnUIReload); void OnUIReload(PanelRenderer renderer, VisualElement root, int version) { if (version == loadedVersion) return; // enable/disable keeps the tree: do not rewire loadedVersion = version; root.dataSource = stats; var hp = root.Q<Game.UI.ResourceBar>("wrist-hp"); hp.SetBinding(nameof(Game.UI.ResourceBar.MaxValue), ToTarget(nameof(PlayerStats.MaxHealth))); hp.SetBinding(nameof(Game.UI.ResourceBar.Value), ToTarget(nameof(PlayerStats.Health))); headingLabel = root.Q<Label>("wrist-heading"); clockLabel = root.Q<Label>("wrist-clock"); // Readouts that are not data-bound: refresh 10x per second, not every frame. root.schedule.Execute(UpdateReadouts).Every(100); UpdateReadouts(); } void UpdateReadouts() { if (viewCamera == null) return; int heading = Mathf.RoundToInt(viewCamera.transform.eulerAngles.y) % 360; headingLabel.text = quot;{heading:000}"; var t = TimeSpan.FromSeconds(Time.timeSinceLevelLoad); clockLabel.text = quot;{(int)t.TotalMinutes:00}:{t.Seconds:00}"; } void Update() { if (viewCamera == null) return; // Dot of the panel's outward normal with the direction to the camera Vector3 normal = flipFacing ? -transform.forward : transform.forward; Vector3 toCamera = viewCamera.transform.position - transform.position; float distance = toCamera.magnitude; float facing = distance > 0.0001f ? Vector3.Dot(normal, toCamera / distance) : 0f; bool shouldShow = distance <= maxDistance && (visible ? facing > hideBelow : facing > showAbove); if (shouldShow != visible) { visible = shouldShow; // 6000.5+: disabling a PanelRenderer keeps its visual tree in memory, so this is cheap. panelRenderer.enabled = visible; } } static DataBinding ToTarget(string path) => new DataBinding { dataSourcePath = new PropertyPath(path), bindingMode = BindingMode.ToTarget }; } ``` Explanation: - **Hysteresis** (`showAbove` > `hideBelow`) stops flicker at the threshold. - Because `Update` disables the same component whose `OnEnable` registers the reload callback, the version guard in `OnUIReload` matters: the callback re-fires on every re-enable with an unchanged version. Without the guard you would stack a new 100 ms scheduler item on each raise of the wrist. - Data (`HP`) is bound; time and heading are polled at 10 Hz via `schedule`. Anything derived from non-notifying sources should be polled at the rate that is perceptible, never per-frame without reason. - Because the panel is unlit and sorted by camera distance, check how it looks inside tight spaces (it can draw over close geometry because no depth test is exposed). If your wrist is sometimes inside the player's own body or gun, adjust the anchor or use a URP Render Objects approach. ## Step B5 - Test checklist 1. Play. Raise the wrist toward the camera; the panel appears, HP matches `PlayerStats`. Use *Damage 10* on `PlayerStats`. 2. Lower the wrist: the panel disappears. Raise it again: no duplicate scheduled work (verify in the Profiler that UI Toolkit update cost does not climb with each raise). 3. Move away > 0.9 m: hidden. 4. Screen-space HUD and wrist show the same health simultaneously because both bind to the same `PlayerStats`. --- # Where to take it next - Add a CRT shader/material for the terminal (scanlines, curvature, flicker, noise), using the render texture as a texture input. UI stays crisp; the look lives in the material. - Add a hover cursor sprite/glow in the 3D world at the hit point for feedback. - Make the wrist interactive (Panel Input Configuration, finger/ray collider) with a few buttons. - Swap the keyboard input in `TerminalUseMode` for an Input Action asset. Next: [[7.UI Toolkit Pitfalls and Performance]].