# 1. Rhythm Design and the Synth Rig Part of [[MIDI Rhythm Game]]. Next: [[2. Conductor and the dspTime Beat Clock]]. ## 1.1 What a rhythm game actually measures A rhythm game compares two clocks: 1. **When the music says a note happens** (the chart, mapped onto audio time). 2. **When the player's input happened** (an input event timestamp). The difference is the **error** in milliseconds, and the judgement (Perfect / Great / Good / Miss) is just a bucket for that error. Everything in this series exists to make both clocks honest: | Problem | Naive approach | What we do instead | |---|---|---| | Song position | `Time.time` since Play | `AudioSettings.dspTime` since a `PlayScheduled` start (part 2) | | Input time | "the frame I noticed the key" | `InputAction.CallbackContext.time` (event timestamp) mapped to song time (part 4) | | Audio you hear vs audio Unity mixed | ignore | a measured output-latency offset (part 6) | | MIDI to hardware | send in `Update` | a dsp-aligned scheduler thread (part 5) | At 60 fps one frame is 16.7 ms, and a "Perfect" window is typically 30-50 ms wide in total. If you judge by frame time you have thrown away a third of your window before the player even touches a key. ## 1.2 Level Up! design notes From [[Level Up! (Review)]]: - **The chart is level design.** Rogers' Level 9 advice about pacing and his beat charts apply directly: rest bars are "safe rooms", dense runs are combat, a key change is a new area. - **Telegraphs.** Notes approach from the far end of a lane for a fixed *time* (not distance). That approach time is the same "fair warning" you gave traps in [[Hazards, Traps and Key-Door Puzzles]]. - **Feedback.** Level 16 says sound is feedback. Hit sounds in a rhythm game are risky, though: a hit sound that is late (because of latency) teaches the player to play late. Prefer visual hit feedback and let the music itself be the reward. - **Difficulty.** Easy/Normal/Hard are separate charts of the same song, not faster scroll speed. Scroll speed is a *readability* setting the player owns (an accessibility option, see [[Accessibility]]). ## 1.3 Two modes, one chart ``` Track mode NoteChart --> RhythmConductor (PlayScheduled backing clip on dspTime) --> NoteHighway (visuals) --> RhythmJudge <-- RhythmInput (keys / pads) Synth mode NoteChart --> RhythmConductor (no clip; dspTime is still the clock) --> ChartMidiSequencer --> MidiEventScheduler thread --> MidiOutService |-- 0xF8 clock (24 ppqn) --> Subharmonicon (tempo follows Unity) |-- note on/off ch 1 --> Mother-32 (bass/melody) --> NoteHighway --> RhythmJudge <-- RhythmInput (keys / Minis MIDI keyboard) Moogs' audio --> your mixer/speakers (monitor directly; optional AudioInputTap for visuals) ``` The chart has **tracks**: one or more **playable lanes** (what the player hits) and **band parts** (what the synths play). In synth mode the Mother-32 can also play the *player's* part as a guide at low velocity on Easy. ## 1.4 Signal map (Marshall's rig) This reuses the wiring from [[Eurorack and Unity]] sections 1.1-1.3: - Unity (RtMidi) -> USB-MIDI cable OUT -> **Mother-32 MIDI IN** (5-pin DIN), channel 1. - Mother-32 has no MIDI THRU, so the **Subharmonicon** needs its own MIDI OUT (a second interface port, or a rack MIDI module that splits). Subharmonicon on channel 2 (set by Moog's SysEx file), MIDI IN on 3.5 mm TRS Type A with the supplied adapter. - A MIDI keyboard or pad controller into Unity via **Minis** for the player's input (the USB cable's MIDI IN, or the controller's own USB). - Synth audio -> your audio interface -> speakers/headphones. **Monitor the synths directly** (hardware monitoring) for the lowest latency. Only route them back through Unity (`AudioInputTap`) for visualisation. > [!tip] One MIDI OUT only? > With a single-output cable, drive the Mother-32 with notes and let the Subharmonicon run on its own internal clock at the same BPM. Synth mode still works because the *Mother-32 notes* define the groove; the Subharmonicon is atmosphere. Add a second port later. ## 1.5 Project layout ``` Assets/ Rhythm/ Scripts/ Core/ RhythmConductor.cs, NoteChart.cs, JudgementWindows.cs Charts/ MidiFileReader.cs Editor/ MidiChartImporterMenu.cs (Editor-only asmdef or Editor folder) Play/ NoteHighway.cs, NoteView.cs, RhythmJudge.cs, RhythmInput.cs, RhythmHud.cs Synth/ MidiEventScheduler.cs, ChartMidiSequencer.cs Calibration/ LatencyCalibrator.cs, RhythmSettings.cs Charts/ *.asset (NoteChart), *.mid.bytes (source MIDI files) Audio/ backing tracks (Load Type: Decompress On Load or Compressed In Memory; not Streaming for tight starts) Scripts/Eurorack/ (from Eurorack and Unity) ``` Import backing tracks as **Decompress On Load** or **Compressed In Memory** with **Preload Audio Data** on, so `PlayScheduled` has the data ready. Streaming clips can start late when the disk is busy. ## 1.6 Exercise Pick a song (your own Mother-32 sequence is ideal) and draw its beat chart on paper: bars, sections, and where the player should rest. Mark two "hero" moments. You will turn this into a chart in part 3.