File size: 21,736 Bytes
f6228f9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
---

comments: true
description: Discover efficient, flexible, and customizable multi-object tracking with Ultralytics YOLO. Learn to track real-time video streams with ease.
keywords: multi-object tracking, Ultralytics YOLO, video analytics, real-time tracking, object detection, AI, machine learning
---


# Multi-Object Tracking with Ultralytics YOLO

<img width="1024" src="https://github.com/ultralytics/docs/releases/download/0/multi-object-tracking-examples.avif" alt="Multi-object tracking examples">

Object tracking in the realm of video analytics is a critical task that not only identifies the location and class of objects within the frame but also maintains a unique ID for each detected object as the video progresses. The applications are limitless—ranging from surveillance and security to real-time sports analytics.

## Why Choose Ultralytics YOLO for Object Tracking?

The output from Ultralytics trackers is consistent with standard [object detection](https://www.ultralytics.com/glossary/object-detection) but has the added value of object IDs. This makes it easy to track objects in video streams and perform subsequent analytics. Here's why you should consider using Ultralytics YOLO for your object tracking needs:

- **Efficiency:** Process video streams in real-time without compromising [accuracy](https://www.ultralytics.com/glossary/accuracy).
- **Flexibility:** Supports multiple tracking algorithms and configurations.
- **Ease of Use:** Simple Python API and CLI options for quick integration and deployment.
- **Customizability:** Easy to use with custom trained YOLO models, allowing integration into domain-specific applications.

<p align="center">
  <br>
  <iframe loading="lazy" width="720" height="405" src="https://www.youtube.com/embed/hHyHmOtmEgs?si=VNZtXmm45Nb9s-N-"

    title="YouTube video player" frameborder="0"

    allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"

    allowfullscreen>
  </iframe>
  <br>
  <strong>Watch:</strong> Object Detection and Tracking with Ultralytics YOLO.
</p>

## Real-world Applications

|           Transportation           |              Retail              |         Aquaculture          |
| :--------------------------------: | :------------------------------: | :--------------------------: |
| ![Vehicle Tracking][vehicle track] | ![People Tracking][people track] | ![Fish Tracking][fish track] |
|          Vehicle Tracking          |         People Tracking          |        Fish Tracking         |

## Features at a Glance

Ultralytics YOLO extends its object detection features to provide robust and versatile object tracking:

- **Real-Time Tracking:** Seamlessly track objects in high-frame-rate videos.
- **Multiple Tracker Support:** Choose from a variety of established tracking algorithms.
- **Customizable Tracker Configurations:** Tailor the tracking algorithm to meet specific requirements by adjusting various parameters.

## Available Trackers

Ultralytics YOLO supports the following tracking algorithms. They can be enabled by passing the relevant YAML configuration file such as `tracker=tracker_type.yaml`:

- [BoT-SORT](https://github.com/NirAharon/BoT-SORT) - Use `botsort.yaml` to enable this tracker.
- [ByteTrack](https://github.com/ifzhang/ByteTrack) - Use `bytetrack.yaml` to enable this tracker.

The default tracker is BoT-SORT.

## Tracking

!!! warning "Tracker Threshold Information"

    If object confidence score will be low, i.e lower than [`track_high_thresh`](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/trackers/bytetrack.yaml#L5), then there will be no tracks successfully returned and updated.


To run the tracker on video streams, use a trained Detect, Segment or Pose model such as YOLO11n, YOLO11n-seg and YOLO11n-pose.

!!! example

    === "Python"


        ```python

        from ultralytics import YOLO


        # Load an official or custom model

        model = YOLO("yolo11n.pt")  # Load an official Detect model

        model = YOLO("yolo11n-seg.pt")  # Load an official Segment model

        model = YOLO("yolo11n-pose.pt")  # Load an official Pose model

        model = YOLO("path/to/best.pt")  # Load a custom trained model


        # Perform tracking with the model

        results = model.track("https://youtu.be/LNwODJXcvt4", show=True)  # Tracking with default tracker

        results = model.track("https://youtu.be/LNwODJXcvt4", show=True, tracker="bytetrack.yaml")  # with ByteTrack

        ```


    === "CLI"


        ```bash

        # Perform tracking with various models using the command line interface

        yolo track model=yolo11n.pt source="https://youtu.be/LNwODJXcvt4"  # Official Detect model

        yolo track model=yolo11n-seg.pt source="https://youtu.be/LNwODJXcvt4"  # Official Segment model

        yolo track model=yolo11n-pose.pt source="https://youtu.be/LNwODJXcvt4"  # Official Pose model

        yolo track model=path/to/best.pt source="https://youtu.be/LNwODJXcvt4"  # Custom trained model


        # Track using ByteTrack tracker

        yolo track model=path/to/best.pt tracker="bytetrack.yaml"

        ```


As can be seen in the above usage, tracking is available for all Detect, Segment and Pose models run on videos or streaming sources.

## Configuration

!!! warning "Tracker Threshold Information"

    If object confidence score will be low, i.e lower than [`track_high_thresh`](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/trackers/bytetrack.yaml#L5), then there will be no tracks successfully returned and updated.


### Tracking Arguments

Tracking configuration shares properties with Predict mode, such as `conf`, `iou`, and `show`. For further configurations, refer to the [Predict](../modes/predict.md#inference-arguments) model page.

!!! example

    === "Python"


        ```python

        from ultralytics import YOLO


        # Configure the tracking parameters and run the tracker

        model = YOLO("yolo11n.pt")

        results = model.track(source="https://youtu.be/LNwODJXcvt4", conf=0.3, iou=0.5, show=True)

        ```


    === "CLI"


        ```bash

        # Configure tracking parameters and run the tracker using the command line interface

        yolo track model=yolo11n.pt source="https://youtu.be/LNwODJXcvt4" conf=0.3, iou=0.5 show

        ```


### Tracker Selection

Ultralytics also allows you to use a modified tracker configuration file. To do this, simply make a copy of a tracker config file (for example, `custom_tracker.yaml`) from [ultralytics/cfg/trackers](https://github.com/ultralytics/ultralytics/tree/main/ultralytics/cfg/trackers) and modify any configurations (except the `tracker_type`) as per your needs.

!!! example

    === "Python"


        ```python

        from ultralytics import YOLO


        # Load the model and run the tracker with a custom configuration file

        model = YOLO("yolo11n.pt")

        results = model.track(source="https://youtu.be/LNwODJXcvt4", tracker="custom_tracker.yaml")

        ```


    === "CLI"


        ```bash

        # Load the model and run the tracker with a custom configuration file using the command line interface

        yolo track model=yolo11n.pt source="https://youtu.be/LNwODJXcvt4" tracker='custom_tracker.yaml'

        ```


For a comprehensive list of tracking arguments, refer to the [ultralytics/cfg/trackers](https://github.com/ultralytics/ultralytics/tree/main/ultralytics/cfg/trackers) page.

## Python Examples

### Persisting Tracks Loop

Here is a Python script using [OpenCV](https://www.ultralytics.com/glossary/opencv) (`cv2`) and YOLO11 to run object tracking on video frames. This script still assumes you have already installed the necessary packages (`opencv-python` and `ultralytics`). The `persist=True` argument tells the tracker that the current image or frame is the next in a sequence and to expect tracks from the previous image in the current image.

!!! example "Streaming for-loop with tracking"

    ```python

    import cv2


    from ultralytics import YOLO


    # Load the YOLO11 model

    model = YOLO("yolo11n.pt")


    # Open the video file

    video_path = "path/to/video.mp4"

    cap = cv2.VideoCapture(video_path)


    # Loop through the video frames

    while cap.isOpened():

        # Read a frame from the video

        success, frame = cap.read()


        if success:

            # Run YOLO11 tracking on the frame, persisting tracks between frames

            results = model.track(frame, persist=True)


            # Visualize the results on the frame

            annotated_frame = results[0].plot()


            # Display the annotated frame

            cv2.imshow("YOLO11 Tracking", annotated_frame)


            # Break the loop if 'q' is pressed

            if cv2.waitKey(1) & 0xFF == ord("q"):

                break

        else:

            # Break the loop if the end of the video is reached

            break


    # Release the video capture object and close the display window

    cap.release()

    cv2.destroyAllWindows()

    ```


Please note the change from `model(frame)` to `model.track(frame)`, which enables object tracking instead of simple detection. This modified script will run the tracker on each frame of the video, visualize the results, and display them in a window. The loop can be exited by pressing 'q'.

### Plotting Tracks Over Time

Visualizing object tracks over consecutive frames can provide valuable insights into the movement patterns and behavior of detected objects within a video. With Ultralytics YOLO11, plotting these tracks is a seamless and efficient process.

In the following example, we demonstrate how to utilize YOLO11's tracking capabilities to plot the movement of detected objects across multiple video frames. This script involves opening a video file, reading it frame by frame, and utilizing the YOLO model to identify and track various objects. By retaining the center points of the detected bounding boxes and connecting them, we can draw lines that represent the paths followed by the tracked objects.

!!! example "Plotting tracks over multiple video frames"

    ```python

    from collections import defaultdict


    import cv2

    import numpy as np


    from ultralytics import YOLO


    # Load the YOLO11 model

    model = YOLO("yolo11n.pt")


    # Open the video file

    video_path = "path/to/video.mp4"

    cap = cv2.VideoCapture(video_path)


    # Store the track history

    track_history = defaultdict(lambda: [])


    # Loop through the video frames

    while cap.isOpened():

        # Read a frame from the video

        success, frame = cap.read()


        if success:

            # Run YOLO11 tracking on the frame, persisting tracks between frames

            results = model.track(frame, persist=True)


            # Get the boxes and track IDs

            boxes = results[0].boxes.xywh.cpu()

            track_ids = results[0].boxes.id.int().cpu().tolist()


            # Visualize the results on the frame

            annotated_frame = results[0].plot()


            # Plot the tracks

            for box, track_id in zip(boxes, track_ids):

                x, y, w, h = box

                track = track_history[track_id]

                track.append((float(x), float(y)))  # x, y center point

                if len(track) > 30:  # retain 90 tracks for 90 frames

                    track.pop(0)


                # Draw the tracking lines

                points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))

                cv2.polylines(annotated_frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)


            # Display the annotated frame

            cv2.imshow("YOLO11 Tracking", annotated_frame)


            # Break the loop if 'q' is pressed

            if cv2.waitKey(1) & 0xFF == ord("q"):

                break

        else:

            # Break the loop if the end of the video is reached

            break


    # Release the video capture object and close the display window

    cap.release()

    cv2.destroyAllWindows()

    ```


### Multithreaded Tracking

Multithreaded tracking provides the capability to run object tracking on multiple video streams simultaneously. This is particularly useful when handling multiple video inputs, such as from multiple surveillance cameras, where concurrent processing can greatly enhance efficiency and performance.

In the provided Python script, we make use of Python's `threading` module to run multiple instances of the tracker concurrently. Each thread is responsible for running the tracker on one video file, and all the threads run simultaneously in the background.

To ensure that each thread receives the correct parameters (the video file, the model to use and the file index), we define a function `run_tracker_in_thread` that accepts these parameters and contains the main tracking loop. This function reads the video frame by frame, runs the tracker, and displays the results.

Two different models are used in this example: `yolo11n.pt` and `yolo11n-seg.pt`, each tracking objects in a different video file. The video files are specified in `video_file1` and `video_file2`.

The `daemon=True` parameter in `threading.Thread` means that these threads will be closed as soon as the main program finishes. We then start the threads with `start()` and use `join()` to make the main thread wait until both tracker threads have finished.

Finally, after all threads have completed their task, the windows displaying the results are closed using `cv2.destroyAllWindows()`.

!!! example "Streaming for-loop with tracking"

    ```python

    import threading


    import cv2


    from ultralytics import YOLO


    # Define model names and video sources

    MODEL_NAMES = ["yolo11n.pt", "yolo11n-seg.pt"]

    SOURCES = ["path/to/video.mp4", "0"]  # local video, 0 for webcam



    def run_tracker_in_thread(model_name, filename):

        """

        Run YOLO tracker in its own thread for concurrent processing.


        Args:

            model_name (str): The YOLO11 model object.

            filename (str): The path to the video file or the identifier for the webcam/external camera source.

        """

        model = YOLO(model_name)

        results = model.track(filename, save=True, stream=True)

        for r in results:

            pass



    # Create and start tracker threads using a for loop

    tracker_threads = []

    for video_file, model_name in zip(SOURCES, MODEL_NAMES):

        thread = threading.Thread(target=run_tracker_in_thread, args=(model_name, video_file), daemon=True)

        tracker_threads.append(thread)

        thread.start()


    # Wait for all tracker threads to finish

    for thread in tracker_threads:

        thread.join()


    # Clean up and close windows

    cv2.destroyAllWindows()

    ```


This example can easily be extended to handle more video files and models by creating more threads and applying the same methodology.

## Contribute New Trackers

Are you proficient in multi-object tracking and have successfully implemented or adapted a tracking algorithm with Ultralytics YOLO? We invite you to contribute to our Trackers section in [ultralytics/cfg/trackers](https://github.com/ultralytics/ultralytics/tree/main/ultralytics/cfg/trackers)! Your real-world applications and solutions could be invaluable for users working on tracking tasks.

By contributing to this section, you help expand the scope of tracking solutions available within the Ultralytics YOLO framework, adding another layer of functionality and utility for the community.

To initiate your contribution, please refer to our [Contributing Guide](../help/contributing.md) for comprehensive instructions on submitting a Pull Request (PR) 🛠️. We are excited to see what you bring to the table!

Together, let's enhance the tracking capabilities of the Ultralytics YOLO ecosystem 🙏!

[fish track]: https://github.com/RizwanMunawar/ultralytics/assets/62513924/a5146d0f-bfa8-4e0a-b7df-3c1446cd8142
[people track]: https://github.com/RizwanMunawar/ultralytics/assets/62513924/93bb4ee2-77a0-4e4e-8eb6-eb8f527f0527
[vehicle track]: https://github.com/RizwanMunawar/ultralytics/assets/62513924/ee6e6038-383b-4f21-ac29-b2a1c7d386ab

## FAQ

### What is Multi-Object Tracking and how does Ultralytics YOLO support it?

Multi-object tracking in video analytics involves both identifying objects and maintaining a unique ID for each detected object across video frames. Ultralytics YOLO supports this by providing real-time tracking along with object IDs, facilitating tasks such as security surveillance and sports analytics. The system uses trackers like BoT-SORT and ByteTrack, which can be configured via YAML files.

### How do I configure a custom tracker for Ultralytics YOLO?

You can configure a custom tracker by copying an existing tracker configuration file (e.g., `custom_tracker.yaml`) from the [Ultralytics tracker configuration directory](https://github.com/ultralytics/ultralytics/tree/main/ultralytics/cfg/trackers) and modifying parameters as needed, except for the `tracker_type`. Use this file in your tracking model like so:

!!! example

    === "Python"


        ```python

        from ultralytics import YOLO


        model = YOLO("yolo11n.pt")

        results = model.track(source="https://youtu.be/LNwODJXcvt4", tracker="custom_tracker.yaml")

        ```


    === "CLI"


        ```bash

        yolo track model=yolo11n.pt source="https://youtu.be/LNwODJXcvt4" tracker='custom_tracker.yaml'

        ```


### How can I run object tracking on multiple video streams simultaneously?

To run object tracking on multiple video streams simultaneously, you can use Python's `threading` module. Each thread will handle a separate video stream. Here's an example of how you can set this up:

!!! example "Multithreaded Tracking"

    ```python

    import threading


    import cv2


    from ultralytics import YOLO


    # Define model names and video sources

    MODEL_NAMES = ["yolo11n.pt", "yolo11n-seg.pt"]

    SOURCES = ["path/to/video.mp4", "0"]  # local video, 0 for webcam



    def run_tracker_in_thread(model_name, filename):

        """

        Run YOLO tracker in its own thread for concurrent processing.


        Args:

            model_name (str): The YOLO11 model object.

            filename (str): The path to the video file or the identifier for the webcam/external camera source.

        """

        model = YOLO(model_name)

        results = model.track(filename, save=True, stream=True)

        for r in results:

            pass



    # Create and start tracker threads using a for loop

    tracker_threads = []

    for video_file, model_name in zip(SOURCES, MODEL_NAMES):

        thread = threading.Thread(target=run_tracker_in_thread, args=(model_name, video_file), daemon=True)

        tracker_threads.append(thread)

        thread.start()


    # Wait for all tracker threads to finish

    for thread in tracker_threads:

        thread.join()


    # Clean up and close windows

    cv2.destroyAllWindows()

    ```


### What are the real-world applications of multi-object tracking with Ultralytics YOLO?

Multi-object tracking with Ultralytics YOLO has numerous applications, including:

- **Transportation:** Vehicle tracking for traffic management and autonomous driving.
- **Retail:** People tracking for in-store analytics and security.
- **Aquaculture:** Fish tracking for monitoring aquatic environments.

These applications benefit from Ultralytics YOLO's ability to process high-frame-rate videos in real time.

### How can I visualize object tracks over multiple video frames with Ultralytics YOLO?

To visualize object tracks over multiple video frames, you can use the YOLO model's tracking features along with OpenCV to draw the paths of detected objects. Here's an example script that demonstrates this:

!!! example "Plotting tracks over multiple video frames"

    ```python

    from collections import defaultdict


    import cv2

    import numpy as np


    from ultralytics import YOLO


    model = YOLO("yolo11n.pt")

    video_path = "path/to/video.mp4"

    cap = cv2.VideoCapture(video_path)

    track_history = defaultdict(lambda: [])


    while cap.isOpened():

        success, frame = cap.read()

        if success:

            results = model.track(frame, persist=True)

            boxes = results[0].boxes.xywh.cpu()

            track_ids = results[0].boxes.id.int().cpu().tolist()

            annotated_frame = results[0].plot()

            for box, track_id in zip(boxes, track_ids):

                x, y, w, h = box

                track = track_history[track_id]

                track.append((float(x), float(y)))

                if len(track) > 30:

                    track.pop(0)

                points = np.hstack(track).astype(np.int32).reshape((-1, 1, 2))

                cv2.polylines(annotated_frame, [points], isClosed=False, color=(230, 230, 230), thickness=10)

            cv2.imshow("YOLO11 Tracking", annotated_frame)

            if cv2.waitKey(1) & 0xFF == ord("q"):

                break

        else:

            break

    cap.release()

    cv2.destroyAllWindows()

    ```


This script will plot the tracking lines showing the movement paths of the tracked objects over time.