File size: 2,424 Bytes
36ff285 2d44f2b 36ff285 2d44f2b 36ff285 5e82d66 36ff285 5e82d66 36ff285 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
---
license: apache-2.0
datasets:
- lerobot/pusht
pipeline_tag: robotics
---
# Model Card for Transformer based Diffusion Policy / PushT
Transformer based Diffusion Policy (as per [Diffusion Policy: Visuomotor Policy
Learning via Action Diffusion](https://arxiv.org/abs/2303.04137)) trained for the `PushT` environment from [gym-pusht](https://github.com/huggingface/gym-pusht).
## How to Get Started with the Model
See the [LeRobot library](https://github.com/huggingface/lerobot) (particularly the [evaluation script](https://github.com/huggingface/lerobot/blob/main/lerobot/scripts/eval.py)) for instructions on how to load and evaluate this model.
## Training Details
The model was trained using [LeRobot's training script](https://github.com/huggingface/lerobot/blob/d747195c5733c4f68d4bfbe62632d6fc1b605712/lerobot/scripts/train.py) and with the [pusht](https://huggingface.co/datasets/lerobot/pusht/tree/v1.3) dataset, using this command:
```bash
python lerobot/scripts/train.py \
hydra.run.dir=outputs/train/diffusion_pusht \
hydra.job.name=diffusion_pusht \
policy=diffusion training.save_model=true \
env=pusht \
env.task=PushT-v0 \
dataset_repo_id=lerobot/pusht \
training.save_freq=25000 \
training.eval_freq=10000 \
wandb.enable=true \
device=cuda
```
The training and eval curves may be found at https://api.wandb.ai/links/none7/rd2trav7
This took about 6 hours to train on an Nvida Tesla P100 GPU.
## Evaluation
The model was evaluated on the `PushT` environment from [gym-pusht](https://github.com/huggingface/gym-pusht) and compared to a similar model trained with the original [Diffusion Policy code](https://github.com/real-stanford/diffusion_policy). There are two evaluation metrics on a per-episode basis:
- Maximum overlap with target (seen as `eval/avg_max_reward` in the charts above). This ranges in [0, 1].
- Success: whether or not the maximum overlap is at least 95%.
Here are the metrics for 500 episodes worth of evaluation. The "Theirs" column is for an equivalent model from the experiments of original Diffusion Policy repository and evaluated on LeRobot (the model weights may be found in the original_dp_repo branch of this respository).
<blank>|Ours|Theirs
-|-|-
Average max. overlap ratio | 0.799 | 0.605
Success rate for 500 episodes (%) | 32.00 | 17.00
The results of each of the individual rollouts may be found in [eval_info.json](eval_info.json). |