File size: 3,907 Bytes
b316424 2a24662 b316424 2a24662 b316424 2a24662 b316424 2a24662 b316424 2a24662 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
---
library_name: ml-agents
tags:
- Pyramids
- deep-reinforcement-learning
- reinforcement-learning
- ML-Agents-Pyramids
license: apache-2.0
---
# **ppo** Agent playing **Pyramids**
This is a trained model of a **ppo** agent playing **Pyramids**
using the [Unity ML-Agents Library](https://github.com/Unity-Technologies/ml-agents).
## Watch the Agent play
You can watch the agent playing directly in your browser
Go to https://huggingface.co/spaces/unity/ML-Agents-Pyramids
Step 1: Find the model_id: Francesco-A/ppo-Pyramids-v1
Step 2: Select the .nn /.onnx file
Click on Watch the agent play
### Resume the training
```bash
mlagents-learn <your_configuration_file_path.yaml> --run-id=<run_id> --resume
```
### Training hyperparameters
```python
behaviors:
Pyramids:
trainer_type: ppo
hyperparameters:
batch_size: 128
buffer_size: 2048
learning_rate: 0.0003
beta: 0.01
epsilon: 0.2
lambd: 0.95
num_epoch: 3
learning_rate_schedule: linear
network_settings:
normalize: false
hidden_units: 512
num_layers: 2
vis_encode_type: simple
reward_signals:
extrinsic:
gamma: 0.99
strength: 1.0
rnd:
gamma: 0.99
strength: 0.01
network_settings:
hidden_units: 64
num_layers: 3
learning_rate: 0.0001
keep_checkpoints: 5
max_steps: 1000000
time_horizon: 128
summary_freq: 30000
```
## Training details
| Step | Time Elapsed | Mean Reward | Std of Reward | Status |
|---------|--------------|-------------|---------------|-----------|
| 30000 | 59.481 s | -1.000 | 0.000 | Training |
| 60000 | 118.648 s | -0.798 | 0.661 | Training |
| 90000 | 180.684 s | -0.701 | 0.808 | Training |
| 120000 | 240.734 s | -0.931 | 0.373 | Training |
| 150000 | 300.978 s | -0.851 | 0.588 | Training |
| 180000 | 360.137 s | -0.934 | 0.361 | Training |
| 210000 | 424.326 s | -1.000 | 0.000 | Training |
| 240000 | 484.774 s | -0.849 | 0.595 | Training |
| 270000 | 546.089 s | -0.377 | 1.029 | Training |
| 300000 | 614.797 s | -0.735 | 0.689 | Training |
| 330000 | 684.241 s | -0.926 | 0.405 | Training |
| 360000 | 745.790 s | -0.819 | 0.676 | Training |
| 390000 | 812.573 s | -0.715 | 0.755 | Training |
| 420000 | 877.836 s | -0.781 | 0.683 | Training |
| 450000 | 944.423 s | -0.220 | 1.114 | Training |
| 480000 | 1010.918 s | -0.484 | 0.962 | Training |
| 510000 | 1074.058 s | -0.003 | 1.162 | Training |
| 540000 | 1138.848 s | -0.021 | 1.222 | Training |
| 570000 | 1204.326 s | 0.384 | 1.231 | Training |
| 600000 | 1276.488 s | 0.690 | 1.174 | Training |
| 630000 | 1345.297 s | 0.943 | 1.058 | Training |
| 660000 | 1412.791 s | 1.014 | 1.043 | Training |
| 690000 | 1482.712 s | 0.927 | 1.054 | Training |
| 720000 | 1548.726 s | 0.900 | 1.128 | Training |
| 750000 | 1618.284 s | 1.379 | 0.701 | Training |
| 780000 | 1692.080 s | 1.567 | 0.359 | Training |
| 810000 | 1762.159 s | 1.475 | 0.567 | Training |
| 840000 | 1832.166 s | 1.438 | 0.648 | Training |
| 870000 | 1907.191 s | 1.534 | 0.536 | Training |
| 900000 | 1977.521 s | 1.552 | 0.478 | Training |
| 930000 | 2051.259 s | 1.458 | 0.633 | Training |
| 960000 | 2126.498 s | 1.545 | 0.586 | Training |
| 990000 | 2198.591 s | 1.565 | 0.591 | Training | |