# 1. ML-Agents Sentis and the Training Loop
Part of [[ML-Agents and Inference Engine]]. Next: [[2. Installing ML-Agents and Python]].
## 1.1 The pieces
```
TRAINING SHIPPING
Unity scene (Editor or a build) Unity scene
Agent components --observations--> Python Agent components
<---actions----- mlagents-learn BehaviorParameters.Model = trained .onnx (ModelAsset)
--rewards------> (PyTorch PPO/SAC) Sentis (com.unity.ai.inference) runs the network
| no Python, no network connection
v
results/<run-id>/<Behavior>.onnx
```
| Piece | What it is | Where it lives |
|---|---|---|
| **Agent** | your C# subclass of `Unity.MLAgents.Agent`: collects observations, applies actions, gives rewards | Unity |
| **Behavior Parameters** | behavior name, observation size, action spec, model, behavior type | Unity component |
| **Decision Requester** | asks for a decision every N physics steps | Unity component |
| **Sensors** | vector observations you write, plus components like Ray Perception Sensor 3D, camera sensors | Unity |
| **Trainer** | `mlagents-learn` (Python package `mlagents`), runs PPO/SAC/POCA with PyTorch, talks to Unity over gRPC | Python |
| **Model** | exported `.onnx`, imported by Sentis as a `ModelAsset` | Unity asset |
| **Sentis** | the inference runtime (`com.unity.ai.inference`); ML-Agents depends on it | Unity package |
The 4.2 manual is explicit that the C# package "does not contain the machine learning algorithms for training"; those are in the Python package.
## 1.2 The names (why this is confusing)
- **Barracuda** was the original inference runtime (3.0.0-exp.1 "Upgraded ML-Agents to Sentis 1.2.0-exp.2 and deprecated Barracuda").
- **Sentis** replaced it (ML-Agents 3.0.0 ended on Sentis 2.1.0).
- The changelog wording is not consistent (4.0.1 and 4.0.2 still say "Inference Engine 2.4.1 / 2.5.0"), so do not read too much into which name a given line uses.
- **Inference Engine** was the 2.2 rename; ML-Agents 4.0.0 "Upgraded to Inference Engine 2.2.1".
- **Sentis again** is the *display name* from 2.4 on (2.6.1 manual: "You can find the Sentis package by searching Sentis or Inference Engine"), still package `com.unity.ai.inference`, still namespace `Unity.InferenceEngine`. ML-Agents 4.0.3 "Upgraded to Sentis 2.6.1".
So: in code you write `using Unity.InferenceEngine;`, in Package Manager you may see "Sentis", and in older forum posts you will see all three names.
## 1.3 Behavior types
`BehaviorParameters.BehaviorType`:
- **Default**: train if a trainer is connected; otherwise run the model if assigned; otherwise run your `Heuristic`.
- **Heuristic Only**: always your `Heuristic` (keyboard control for testing, or a scripted baseline).
- **Inference Only**: always the model (what you ship).
## 1.4 When to use ML (and when not)
| Use ML-Agents when | Use an FSM / behavior tree when |
|---|---|
| The *goal* is easy to state but the *how* is hard to write (balance, racing lines, dodging) | You can describe the behaviour in a few states and transitions |
| You want variety and surprise (sparring partners, AI racers, a "ghost" opponent) | Designers need precise control and readability (bosses, stealth guards) |
| You want automated playtesting of reachability | You need predictable, debuggable behaviour for a class demo |
Rogers' enemy chapter (Level 11 in [[Level Up! (Review)]]) prizes enemies whose patterns the player can learn. A trained agent can be wrapped so that it still telegraphs: the FSM from [[Enemy AI State Machine]] decides *when* the agent attacks and plays the wind-up; the network decides *how it moves* during the chase.
## 1.5 The reward is the design document
In ML-Agents you do not write behaviour, you write **incentives**. Bad incentives produce "reward hacking": the agent finds a loophole you did not intend (spinning in place to farm a shaping reward, ending the episode early to avoid a time penalty). Part 3 shows a reward scheme and part 7 lists the classic hacks. This is a great class discussion: it is the same problem as designing a scoring system players will exploit.
## Sources
- https://docs.unity3d.com/Packages/
[email protected]/manual/index.html
- https://docs.unity3d.com/Packages/
[email protected]/changelog/CHANGELOG.html
- https://docs.unity3d.com/Packages/
[email protected]/manual/index.html