Unity ML-Agents, is an open source toolkit developed by Unity to enhance a game’s AI with machine learning. Typically when developing an AI for a game, you’d check to see if a certain condition is true (i.e. can you see the player?) and then execute a certain action (i.e. attack). This form of AI works, but at the core of things it can be predictable and limiting.
Machine learning allows agents (enemy, AI car, anything you want to have an AI) to automatically learn through reinforcement learning, imitation learning and many other learning types. What this means, is that you’re not specifically telling the agent what to do. Instead, you’re developing their brain overtime in order for them to determine how to go about a certain task with a number of given inputs.
Let’s Look at an Example
Let’s go over an example of training an ML agent (this is from a Unity ML-Agents sample project). We have an agent who can move and turn on a flat surface. Their objective is to push a block into the end goal. We can train this agent’s brain, so that no matter the starting position of the block or goal, it will always be able to complete it. The agent can also detect the surrounding world with 14 raycasts shooting out from all directions. This can give info about what the agent can see and how far away.
Most likely, the first session will start with the agent standing still or moving around in a random direction. If we continue to run these simulations many, many times, eventually the agent will hit the block and maybe even accidentally move it into the goal. This is where agent rewards come in handy. Whenever the agent does something that progresses its learning (i.e. moving the block into the goal), we give it +1 reward. In general, this means the rewarded behaviour will carry over to future simulations and overtime the agent will gather more knowledge on where to stand relative to the block, the direction it needs to push, etc. After hundreds (maybe even thousands) of simulations the agent’s brain should be developed enough so that it can push the block (no matter the starting position) into the goal every time.
Here’s a look at the push block example we just went over.
ML-Agents Package
Right now, ML-Agents is still in development but it can be downloaded from the Unity GitHub page here. Furthermore, since it’s still in development and since the training is done through Python, there are additional things you need to download and setup but the provided tutorials will go through that. ML-Agents also includes 15+ example environments, showing many different game types and how those are trained.
Resource Links
Tutorials
- An Introduction to Unity’s ML-Agents by Tim Bonzon
- How to Train a Machine Learning Agents via Demonstration by Tim Bonzon
- How to Make AIs Target Objects with Unity ML Agents by Tim Bonzon
- Unity ML-Agents Setup – by Immersive Limit
- Unity ML-Agents Tutorial | AI Truffle Pigs! – by Immersive Limit
- Unity ML-Agents Tutorial – Penguins – by Immersive Limit
- How to Install Unity ML Agents – by Philipp Dominic Siedler
- Machine Learning Agents – by Unity
- Kart Racing Game with Machine Learning in Unity! – By Unity
Demos
- AI Flight with Unity ML-Agents – by Immersive Limit
- Explorer Drone | Unity Machine Learning Demo – by m baske
- Angry AI v1 – Battle Robots | Unity Machine Learning Demo – by m baske
- AI Learned How to Play Game | Reinforcement Learning | ML Agents in Unity – by wideflat
Other New Unity Features