|
---
|
|
comments: true
|
|
description: Learn how to optimize YOLOv5 hyperparameters using genetic algorithms for improved training performance. Step-by-step instructions included.
|
|
keywords: YOLOv5, hyperparameter evolution, genetic algorithms, machine learning, optimization, Ultralytics
|
|
---
|
|
|
|
📚 This guide explains **hyperparameter evolution** for YOLOv5 🚀. Hyperparameter evolution is a method of [Hyperparameter Optimization](https://en.wikipedia.org/wiki/Hyperparameter_optimization) using a [Genetic Algorithm](https://en.wikipedia.org/wiki/Genetic_algorithm) (GA) for optimization.
|
|
|
|
Hyperparameters in ML control various aspects of training, and finding optimal values for them can be a challenge. Traditional methods like grid searches can quickly become intractable due to 1) the high dimensional search space 2) unknown correlations among the dimensions, and 3) expensive nature of evaluating the fitness at each point, making GA a suitable candidate for hyperparameter searches.
|
|
|
|
## Before You Start
|
|
|
|
Clone repo and install [requirements.txt](https://github.com/ultralytics/yolov5/blob/master/requirements.txt) in a [**Python>=3.8.0**](https://www.python.org/) environment, including [**PyTorch>=1.8**](https://pytorch.org/get-started/locally/). [Models](https://github.com/ultralytics/yolov5/tree/master/models) and [datasets](https://github.com/ultralytics/yolov5/tree/master/data) download automatically from the latest YOLOv5 [release](https://github.com/ultralytics/yolov5/releases).
|
|
|
|
```bash
|
|
git clone https://github.com/ultralytics/yolov5 # clone
|
|
cd yolov5
|
|
pip install -r requirements.txt # install
|
|
```
|
|
|
|
## 1. Initialize Hyperparameters
|
|
|
|
YOLOv5 has about 30 hyperparameters used for various training settings. These are defined in `*.yaml` files in the `/data/hyps` directory. Better initial guesses will produce better final results, so it is important to initialize these values properly before evolving. If in doubt, simply use the default values, which are optimized for YOLOv5 COCO training from scratch.
|
|
|
|
```yaml
|
|
# YOLOv5 🚀 by Ultralytics, AGPL-3.0 license
|
|
# Hyperparameters for low-augmentation COCO training from scratch
|
|
# python train.py --batch 64 --cfg yolov5n6.yaml --weights '' --data coco.yaml --img 640 --epochs 300 --linear
|
|
# See tutorials for hyperparameter evolution https://github.com/ultralytics/yolov5#tutorials
|
|
|
|
lr0: 0.01 # initial learning rate (SGD=1E-2, Adam=1E-3)
|
|
lrf: 0.01 # final OneCycleLR learning rate (lr0 * lrf)
|
|
momentum: 0.937 # SGD momentum/Adam beta1
|
|
weight_decay: 0.0005 # optimizer weight decay 5e-4
|
|
warmup_epochs: 3.0 # warmup epochs (fractions ok)
|
|
warmup_momentum: 0.8 # warmup initial momentum
|
|
warmup_bias_lr: 0.1 # warmup initial bias lr
|
|
box: 0.05 # box loss gain
|
|
cls: 0.5 # cls loss gain
|
|
cls_pw: 1.0 # cls BCELoss positive_weight
|
|
obj: 1.0 # obj loss gain (scale with pixels)
|
|
obj_pw: 1.0 # obj BCELoss positive_weight
|
|
iou_t: 0.20 # IoU training threshold
|
|
anchor_t: 4.0 # anchor-multiple threshold
|
|
# anchors: 3 # anchors per output layer (0 to ignore)
|
|
fl_gamma: 0.0 # focal loss gamma (efficientDet default gamma=1.5)
|
|
hsv_h: 0.015 # image HSV-Hue augmentation (fraction)
|
|
hsv_s: 0.7 # image HSV-Saturation augmentation (fraction)
|
|
hsv_v: 0.4 # image HSV-Value augmentation (fraction)
|
|
degrees: 0.0 # image rotation (+/- deg)
|
|
translate: 0.1 # image translation (+/- fraction)
|
|
scale: 0.5 # image scale (+/- gain)
|
|
shear: 0.0 # image shear (+/- deg)
|
|
perspective: 0.0 # image perspective (+/- fraction), range 0-0.001
|
|
flipud: 0.0 # image flip up-down (probability)
|
|
fliplr: 0.5 # image flip left-right (probability)
|
|
mosaic: 1.0 # image mosaic (probability)
|
|
mixup: 0.0 # image mixup (probability)
|
|
copy_paste: 0.0 # segment copy-paste (probability)
|
|
```
|
|
|
|
## 2. Define Fitness
|
|
|
|
Fitness is the value we seek to maximize. In YOLOv5 we define a default fitness function as a weighted combination of metrics: `[email protected]` contributes 10% of the weight and `[email protected]:0.95` contributes the remaining 90%, with [Precision `P` and [Recall](https://www.ultralytics.com/glossary/recall) `R`](https://en.wikipedia.org/wiki/Precision_and_recall) absent. You may adjust these as you see fit or use the default fitness definition in utils/metrics.py (recommended).
|
|
|
|
```python
|
|
def fitness(x):
|
|
"""Return model fitness as the sum of weighted metrics [P, R, [email protected], [email protected]:0.95]."""
|
|
w = [0.0, 0.0, 0.1, 0.9] # weights for [P, R, [email protected], [email protected]:0.95]
|
|
return (x[:, :4] * w).sum(1)
|
|
```
|
|
|
|
## 3. Evolve
|
|
|
|
Evolution is performed about a base scenario which we seek to improve upon. The base scenario in this example is [finetuning](https://www.ultralytics.com/glossary/fine-tuning) COCO128 for 10 [epochs](https://www.ultralytics.com/glossary/epoch) using pretrained YOLOv5s. The base scenario training command is:
|
|
|
|
```bash
|
|
python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache
|
|
```
|
|
|
|
To evolve hyperparameters **specific to this scenario**, starting from our initial values defined in **Section 1.**, and maximizing the fitness defined in **Section 2.**, append `--evolve`:
|
|
|
|
```bash
|
|
# Single-GPU
|
|
python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache --evolve
|
|
|
|
# Multi-GPU
|
|
for i in 0 1 2 3 4 5 6 7; do
|
|
sleep $(expr 30 \* $i) && # 30-second delay (optional)
|
|
echo 'Starting GPU '$i'...' &&
|
|
nohup python train.py --epochs 10 --data coco128.yaml --weights yolov5s.pt --cache --device $i --evolve > evolve_gpu_$i.log &
|
|
done
|
|
|
|
# Multi-GPU bash-while (not recommended)
|
|
for i in 0 1 2 3 4 5 6 7; do
|
|
sleep $(expr 30 \* $i) && # 30-second delay (optional)
|
|
echo 'Starting GPU '$i'...' &&
|
|
"$(while true; do nohup python train.py... --device $i --evolve 1 > evolve_gpu_$i.log; done)" &
|
|
done
|
|
```
|
|
|
|
The default evolution settings will run the base scenario 300 times, i.e. for 300 generations. You can modify generations via the `--evolve` argument, i.e. `python train.py --evolve 1000`.
|
|
|
|
The main genetic operators are **crossover** and **mutation**. In this work mutation is used, with an 80% probability and a 0.04 variance to create new offspring based on a combination of the best parents from all previous generations. Results are logged to `runs/evolve/exp/evolve.csv`, and the highest fitness offspring is saved every generation as `runs/evolve/hyp_evolved.yaml`:
|
|
|
|
```yaml
|
|
# YOLOv5 Hyperparameter Evolution Results
|
|
# Best generation: 287
|
|
# Last generation: 300
|
|
# metrics/precision, metrics/recall, metrics/mAP_0.5, metrics/mAP_0.5:0.95, val/box_loss, val/obj_loss, val/cls_loss
|
|
# 0.54634, 0.55625, 0.58201, 0.33665, 0.056451, 0.042892, 0.013441
|
|
|
|
lr0: 0.01 # initial learning rate (SGD=1E-2, Adam=1E-3)
|
|
lrf: 0.2 # final OneCycleLR learning rate (lr0 * lrf)
|
|
momentum: 0.937 # SGD momentum/Adam beta1
|
|
weight_decay: 0.0005 # optimizer weight decay 5e-4
|
|
warmup_epochs: 3.0 # warmup epochs (fractions ok)
|
|
warmup_momentum: 0.8 # warmup initial momentum
|
|
warmup_bias_lr: 0.1 # warmup initial bias lr
|
|
box: 0.05 # box loss gain
|
|
cls: 0.5 # cls loss gain
|
|
cls_pw: 1.0 # cls BCELoss positive_weight
|
|
obj: 1.0 # obj loss gain (scale with pixels)
|
|
obj_pw: 1.0 # obj BCELoss positive_weight
|
|
iou_t: 0.20 # IoU training threshold
|
|
anchor_t: 4.0 # anchor-multiple threshold
|
|
# anchors: 3 # anchors per output layer (0 to ignore)
|
|
fl_gamma: 0.0 # focal loss gamma (efficientDet default gamma=1.5)
|
|
hsv_h: 0.015 # image HSV-Hue augmentation (fraction)
|
|
hsv_s: 0.7 # image HSV-Saturation augmentation (fraction)
|
|
hsv_v: 0.4 # image HSV-Value augmentation (fraction)
|
|
degrees: 0.0 # image rotation (+/- deg)
|
|
translate: 0.1 # image translation (+/- fraction)
|
|
scale: 0.5 # image scale (+/- gain)
|
|
shear: 0.0 # image shear (+/- deg)
|
|
perspective: 0.0 # image perspective (+/- fraction), range 0-0.001
|
|
flipud: 0.0 # image flip up-down (probability)
|
|
fliplr: 0.5 # image flip left-right (probability)
|
|
mosaic: 1.0 # image mosaic (probability)
|
|
mixup: 0.0 # image mixup (probability)
|
|
copy_paste: 0.0 # segment copy-paste (probability)
|
|
```
|
|
|
|
We recommend a minimum of 300 generations of evolution for best results. Note that **evolution is generally expensive and time-consuming**, as the base scenario is trained hundreds of times, possibly requiring hundreds or thousands of GPU hours.
|
|
|
|
## 4. Visualize
|
|
|
|
`evolve.csv` is plotted as `evolve.png` by `utils.plots.plot_evolve()` after evolution finishes with one subplot per hyperparameter showing fitness (y-axis) vs hyperparameter values (x-axis). Yellow indicates higher concentrations. Vertical distributions indicate that a parameter has been disabled and does not mutate. This is user selectable in the `meta` dictionary in train.py, and is useful for fixing parameters and preventing them from evolving.
|
|
|
|
|
|
|
|
## Supported Environments
|
|
|
|
Ultralytics provides a range of ready-to-use environments, each pre-installed with essential dependencies such as [CUDA](https://developer.nvidia.com/cuda-zone), [CUDNN](https://developer.nvidia.com/cudnn), [Python](https://www.python.org/), and [PyTorch](https://pytorch.org/), to kickstart your projects.
|
|
|
|
- **Free GPU Notebooks**: <a href="https://bit.ly/yolov5-paperspace-notebook"><img src="https://assets.paperspace.io/img/gradient-badge.svg" alt="Run on Gradient"></a> <a href="https://colab.research.google.com/github/ultralytics/yolov5/blob/master/tutorial.ipynb"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"></a> <a href="https://www.kaggle.com/ultralytics/yolov5"><img src="https://kaggle.com/static/images/open-in-kaggle.svg" alt="Open In Kaggle"></a>
|
|
- **Google Cloud**: [GCP Quickstart Guide](../environments/google_cloud_quickstart_tutorial.md)
|
|
- **Amazon**: [AWS Quickstart Guide](../environments/aws_quickstart_tutorial.md)
|
|
- **Azure**: [AzureML Quickstart Guide](../environments/azureml_quickstart_tutorial.md)
|
|
- **Docker**: [Docker Quickstart Guide](../environments/docker_image_quickstart_tutorial.md) <a href="https://hub.docker.com/r/ultralytics/yolov5"><img src="https://img.shields.io/docker/pulls/ultralytics/yolov5?logo=docker" alt="Docker Pulls"></a>
|
|
|
|
## Project Status
|
|
|
|
<a href="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml"><img src="https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg" alt="YOLOv5 CI"></a>
|
|
|
|
This badge indicates that all [YOLOv5 GitHub Actions](https://github.com/ultralytics/yolov5/actions) Continuous Integration (CI) tests are successfully passing. These CI tests rigorously check the functionality and performance of YOLOv5 across various key aspects: [training](https://github.com/ultralytics/yolov5/blob/master/train.py), [validation](https://github.com/ultralytics/yolov5/blob/master/val.py), [inference](https://github.com/ultralytics/yolov5/blob/master/detect.py), [export](https://github.com/ultralytics/yolov5/blob/master/export.py), and [benchmarks](https://github.com/ultralytics/yolov5/blob/master/benchmarks.py). They ensure consistent and reliable operation on macOS, Windows, and Ubuntu, with tests conducted every 24 hours and upon each new commit.
|
|
|
