# 6. Latency Calibration and Worked Example
Part of [[MIDI Rhythm Game]]. Prev: [[5. Chart-Driven MIDI Out to the Moogs]]. Next: [[7. Rhythm Game Pitfalls]].
## 6.1 Three latencies, two knobs
| Latency | Where it comes from | Knob |
|---|---|---|
| **Audio output** | Unity's mixer buffers, the OS audio stack, your interface driver | `RhythmConductor.AudioLatencySec` (track mode) |
| **Synth path** | MIDI transport, the synth's response, analog path to speakers | `AudioLatencySec` (synth-mode profile) and/or `MidiEventScheduler.sendEarlySec` |
| **Display** | render queue, monitor/TV processing | highway visual offset (draw notes slightly earlier) |
| Input | USB polling, OS, event processing | absorbed into the audio calibration below |
You cannot measure output latency from code reliably (the OS does not tell Unity what the DAC is doing). The standard rhythm-game answer is to ask the player: **tap along to a click, take the median error.** Because the player taps to what they *hear*, the measured offset includes output latency and input latency together, which is exactly what judgement needs.
## 6.2 `RhythmSettings.cs` (persisted)
```csharp
using UnityEngine;
/// <summary>Per-machine rhythm settings in PlayerPrefs (see the PlayerPrefs notes for the wrapper pattern). NOT compiled.</summary>
public static class RhythmSettings
{
const string KTrack = "rhythm.latency.track";
const string KSynth = "rhythm.latency.synth";
const string KVisual = "rhythm.visualOffset";
const string KApproach = "rhythm.approachSec";
public static double TrackLatencySec { get => PlayerPrefs.GetFloat(KTrack, 0.05f); set => Set(KTrack, value); }
public static double SynthLatencySec { get => PlayerPrefs.GetFloat(KSynth, 0.01f); set => Set(KSynth, value); }
public static double VisualOffsetSec { get => PlayerPrefs.GetFloat(KVisual, 0f); set => Set(KVisual, value); }
public static float ApproachSec { get => PlayerPrefs.GetFloat(KApproach, 1.5f); set { PlayerPrefs.SetFloat(KApproach, value); PlayerPrefs.Save(); } }
static void Set(string key, double v) { PlayerPrefs.SetFloat(key, (float)v); PlayerPrefs.Save(); }
}
```
Latency is a property of *this PC and this audio interface*, so it belongs in local preferences, not the cloud save ([[PlayerPrefs Advanced]], [[Cloud Save and Unity Services]]). `float` is fine here; these are small numbers.
## 6.3 `LatencyCalibrator.cs`
Track mode: clicks come from Unity (`PlayScheduled` on dsp beat times). Synth mode: clicks come from the Mother-32 through the scheduler. Either way the player taps one key on the beat.
```csharp
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.InputSystem;
/// <summary>
/// Plays a click every beat and measures the player's tap error (median, outliers dropped).
/// Result = how much later the player hears/feels the beat than the mixed dspTime. NOT compiled.
/// </summary>
public sealed class LatencyCalibrator : MonoBehaviour
{
public enum Target { TrackMode, SynthMode }
[SerializeField] Target target = Target.TrackMode;
[SerializeField] RhythmConductor conductor; // used only as a clock here
[SerializeField] InputActionReference tapAction;
[SerializeField] double bpm = 100;
[SerializeField] int taps = 24;
[SerializeField] int ignoreFirst = 4;
[Header("Track mode click")]
[SerializeField] AudioClip click;
[SerializeField] AudioSource[] clickVoices = new AudioSource[4]; // round-robin so clicks never cut each other
[Header("Synth mode click")]
[SerializeField] MidiEventScheduler scheduler;
[SerializeField, Range(1, 16)] int mother32Channel = 1;
[SerializeField] int clickNote = 72;
public event System.Action<double> Finished; // seconds
readonly List<double> _errors = new();
int _nextBeat, _voice;
bool _running;
public void Begin()
{
_errors.Clear();
conductor.AudioLatencySec = 0; // measure raw
conductor.SetBpm(bpm);
conductor.SetFirstBeatOffset(0);
conductor.StartSong(leadInSec: 1.0);
_nextBeat = 0;
_running = true;
tapAction.action.performed += OnTap;
tapAction.action.Enable();
}
void OnDisable()
{
if (tapAction != null) tapAction.action.performed -= OnTap;
}
void Update()
{
if (!_running) return;
double spb = 60.0 / bpm;
// Schedule clicks up to 0.3 s ahead.
while (_nextBeat * spb <= conductor.SongTime + 0.3)
{
double dsp = conductor.DspAtSongTime(_nextBeat * spb);
if (target == Target.TrackMode)
{
var v = clickVoices[_voice++ % clickVoices.Length];
v.clip = click;
v.PlayScheduled(dsp);
}
else
{
byte ch = (byte)(mother32Channel - 1);
scheduler.Schedule3(dsp, (byte)(0x90 | ch), clickNote, 110);
scheduler.Schedule3(dsp + 0.05, (byte)(0x80 | ch), clickNote, 0);
}
_nextBeat++;
}
}
void OnTap(InputAction.CallbackContext ctx)
{
if (!_running) return;
double songTime = conductor.SongTimeAtRealtime(ctx.time);
double spb = 60.0 / bpm;
double nearestBeat = System.Math.Round(songTime / spb);
if (nearestBeat < 0) return;
double error = songTime - nearestBeat * spb; // positive = tapped after the mixed beat
_errors.Add(error);
if (_errors.Count >= taps + ignoreFirst) Finish();
}
void Finish()
{
_running = false;
tapAction.action.performed -= OnTap;
conductor.StopSong();
if (scheduler != null) scheduler.ClearPending();
var usable = _errors.GetRange(ignoreFirst, _errors.Count - ignoreFirst);
usable.Sort();
double median = usable[usable.Count / 2];
// Drop taps more than 120 ms from the median (distracted taps), then average the rest.
double sum = 0; int n = 0;
foreach (var e in usable) if (System.Math.Abs(e - median) <= 0.12) { sum += e; n++; }
double result = n > 0 ? sum / n : median;
if (target == Target.TrackMode) RhythmSettings.TrackLatencySec = result;
else RhythmSettings.SynthLatencySec = result;
Debug.Log(
quot;[Calibrate] {target}: {result * 1000.0:0.0} ms from {n} taps (median {median * 1000.0:0.0} ms)");
Finished?.Invoke(result);
}
}
```
**Reading the result.** On a typical Windows setup with a USB interface, track mode often lands somewhere in the tens of milliseconds and synth mode lower, because the Moogs bypass Unity's output buffer. Do not trust my "typical": your number is the truth. If you see a *negative* synth-mode value, the player is anticipating (tapping ahead of the click); run it again with eyes closed.
**Visual offset (optional).** Run the same idea with the sound muted: flash a lane on each beat and have the player tap to the flash. The difference between the visual median and the audio median is display latency. Apply it by drawing the highway slightly earlier: in `NoteHighway`, use `now + visualOffset` for positions only (never for judgement).
## 6.4 Worked example: "Mother Groove", four lanes
### Scene checklist
1. **Packages:** Input System (6000.5 ships 1.20.1), RtMidi and Minis from the Keijiro scoped registry ([[Eurorack and Unity]] section 2).
2. **Input actions:** an action map `Rhythm` with `Lane0..Lane3` (Button: D, F, J, K and gamepad face buttons) and `Tap` (Space). Generate `InputActionReference`s.
3. **Synth Rig** prefab from the Eurorack note (disable `MidiClockThread`).
4. **Rhythm** GameObject with: `RhythmConductor`, `RhythmJudge`, `NoteHighway`, `RhythmInput`, `MidiEventScheduler`, `ChartMidiSequencer`, `RhythmGameBootstrap`.
5. **Highway geometry:** a long dark quad, a `HitLine` empty at the near end, a `SpawnPoint` empty 20 m away, four lane markers. `NoteView` prefab = a flattened cube with a child `Tail` cube (pivot at its near end).
6. **HUD:** `UIDocument` with `score`, `combo`, `judgement` labels plus `RhythmHud`.
7. **Chart:** export `MotherGroove.mid` from your DAW with tracks `PLAY`, `M32`, `SUB`; rename to `MotherGroove.mid.bytes`; right-click > **Rhythm > Create Chart From MIDI**. For track mode, also bounce the song to WAV and assign it to `backingTrack`.
### `RhythmGameBootstrap.cs`
```csharp
using UnityEngine;
/// <summary>Loads a chart into every system, applies calibration, runs track or synth mode. NOT compiled.</summary>
public sealed class RhythmGameBootstrap : MonoBehaviour
{
public enum Mode { Track, Synth }
[SerializeField] Mode mode = Mode.Synth;
[SerializeField] NoteChart chart;
[SerializeField] RhythmConductor conductor;
[SerializeField] AudioSource music; // track mode
[SerializeField] RhythmJudge judge;
[SerializeField] NoteHighway highway;
[SerializeField] ChartMidiSequencer sequencer; // synth mode
[SerializeField] double leadInSec = 2.5;
void Start()
{
judge.Load(chart);
highway.Load(chart);
highway.ApproachSec = RhythmSettings.ApproachSec;
conductor.SetBpm(chart.InitialBpm);
conductor.SetFirstBeatOffset(chart.firstBeatOffsetSec);
double songLength = 0;
foreach (var n in judge.AllNotes) songLength = System.Math.Max(songLength, n.endSec);
songLength += 2.0; // tail room for results
if (mode == Mode.Track)
{
music.clip = chart.backingTrack;
if (music.clip.loadState != AudioDataLoadState.Loaded) music.clip.LoadAudioData();
conductor.AudioLatencySec = RhythmSettings.TrackLatencySec;
if (sequencer != null) sequencer.enabled = false;
}
else
{
music.clip = null; // conductor plays nothing; dspTime still runs
conductor.AudioLatencySec = RhythmSettings.SynthLatencySec;
sequencer.enabled = true;
sequencer.Load(chart);
}
conductor.SongFinished += OnFinished;
conductor.StartSong(leadInSec, songLength);
}
void OnFinished()
{
conductor.SongFinished -= OnFinished;
if (sequencer != null) sequencer.StopAndSilence();
int rank = HighScoreTable.Submit(quot;{chart.songTitle}/{chart.difficulty}", "Player", judge.Score);
PlayerStats.Instance?.SetMax(StatKeys.BestCombo, judge.MaxCombo);
Debug.Log(quot;[Rhythm] {chart.songTitle}: {judge.Score} pts, max combo {judge.MaxCombo}, rank {rank}. " +
quot;P/G/G/M = {judge.Counts[0]}/{judge.Counts[1]}/{judge.Counts[2]}/{judge.Counts[3]}");
}
}
```
> [!note] `music` must be the conductor's AudioSource
> Assign the **same** `AudioSource` to `RhythmConductor.music` and to the bootstrap, so `StartSong` schedules the clip the bootstrap assigned. In synth mode the clip is `null` and the conductor skips scheduling.
### Play test order
1. Calibrate track mode with the click; then synth mode with the Mother-32 click.
2. Track mode first (no hardware variables), Easy windows, `approachSec` 2.0.
3. Synth mode: confirm the Subharmonicon's tempo LED follows Unity during the pre-roll; then play.
4. Watch the early/late readout. A consistent bias of more than ~10 ms means recalibrate; a bias that grows over the song means a tempo-map or BPM mismatch between the chart and the audio.
## 6.5 Classroom variant
Run it as a two-player duet: one student plays the `PLAY` lanes on the keyboard, a second plays a MIDI keyboard whose notes are judged against the `M32` track (load a second `RhythmJudge` with that track treated as playable). Discuss which part *felt* harder and why (Level Up! Level 14, multiplayer, and Level 12, mechanics).
## Sources
- `AudioSource.PlayScheduled`: https://docs.unity3d.com/6000.5/Documentation/ScriptReference/AudioSource.PlayScheduled.html
- `PlayerPrefs`: https://docs.unity3d.com/6000.5/Documentation/ScriptReference/PlayerPrefs.html
- `AudioClip.LoadAudioData`: https://docs.unity3d.com/6000.5/Documentation/ScriptReference/AudioClip.LoadAudioData.html