---
library_name: ml-agents
tags:
- Pyramids
- deep-reinforcement-learning
- reinforcement-learning
- ML-Agents-Pyramids
license: apache-2.0
---

  # **ppo** Agent playing **Pyramids**
  This is a trained model of a **ppo** agent playing **Pyramids**
  using the [Unity ML-Agents Library](https://github.com/Unity-Technologies/ml-agents).

## Watch the Agent play
You can watch the agent playing directly in your browser

Go to https://huggingface.co/spaces/unity/ML-Agents-Pyramids
Step 1: Find the model_id: Francesco-A/ppo-Pyramids-v1
Step 2: Select the .nn /.onnx file
Click on Watch the agent play

  ### Resume the training
  ```bash
  mlagents-learn <your_configuration_file_path.yaml> --run-id=<run_id> --resume
  ```

### Training hyperparameters
```python
behaviors:
  Pyramids:
    trainer_type: ppo
    hyperparameters:
      batch_size: 128
      buffer_size: 2048
      learning_rate: 0.0003
      beta: 0.01
      epsilon: 0.2
      lambd: 0.95
      num_epoch: 3
      learning_rate_schedule: linear
    network_settings:
      normalize: false
      hidden_units: 512
      num_layers: 2
      vis_encode_type: simple
    reward_signals:
      extrinsic:
        gamma: 0.99
        strength: 1.0
      rnd:
        gamma: 0.99
        strength: 0.01
        network_settings:
          hidden_units: 64
          num_layers: 3
        learning_rate: 0.0001
    keep_checkpoints: 5
    max_steps: 1000000
    time_horizon: 128
    summary_freq: 30000
```

## Training details
| Step    | Time Elapsed | Mean Reward | Std of Reward | Status    |
|---------|--------------|-------------|---------------|-----------|
| 30000   | 59.481 s     | -1.000      | 0.000         | Training  |
| 60000   | 118.648 s    | -0.798      | 0.661         | Training  |
| 90000   | 180.684 s    | -0.701      | 0.808         | Training  |
| 120000  | 240.734 s    | -0.931      | 0.373         | Training  |
| 150000  | 300.978 s    | -0.851      | 0.588         | Training  |
| 180000  | 360.137 s    | -0.934      | 0.361         | Training  |
| 210000  | 424.326 s    | -1.000      | 0.000         | Training  |
| 240000  | 484.774 s    | -0.849      | 0.595         | Training  |
| 270000  | 546.089 s    | -0.377      | 1.029         | Training  |
| 300000  | 614.797 s    | -0.735      | 0.689         | Training  |
| 330000  | 684.241 s    | -0.926      | 0.405         | Training  |
| 360000  | 745.790 s    | -0.819      | 0.676         | Training  |
| 390000  | 812.573 s    | -0.715      | 0.755         | Training  |
| 420000  | 877.836 s    | -0.781      | 0.683         | Training  |
| 450000  | 944.423 s    | -0.220      | 1.114         | Training  |
| 480000  | 1010.918 s   | -0.484      | 0.962         | Training  |
| 510000  | 1074.058 s   | -0.003      | 1.162         | Training  |
| 540000  | 1138.848 s   | -0.021      | 1.222         | Training  |
| 570000  | 1204.326 s   | 0.384       | 1.231         | Training  |
| 600000  | 1276.488 s   | 0.690       | 1.174         | Training  |
| 630000  | 1345.297 s   | 0.943       | 1.058         | Training  |
| 660000  | 1412.791 s   | 1.014       | 1.043         | Training  |
| 690000  | 1482.712 s   | 0.927       | 1.054         | Training  |
| 720000  | 1548.726 s   | 0.900       | 1.128         | Training  |
| 750000  | 1618.284 s   | 1.379       | 0.701         | Training  |
| 780000  | 1692.080 s   | 1.567       | 0.359         | Training  |
| 810000  | 1762.159 s   | 1.475       | 0.567         | Training  |
| 840000  | 1832.166 s   | 1.438       | 0.648         | Training  |
| 870000  | 1907.191 s   | 1.534       | 0.536         | Training  |
| 900000  | 1977.521 s   | 1.552       | 0.478         | Training  |
| 930000  | 2051.259 s   | 1.458       | 0.633         | Training  |
| 960000  | 2126.498 s   | 1.545       | 0.586         | Training  |
| 990000  | 2198.591 s   | 1.565       | 0.591         | Training  |