Hugging Face's logo

qualcomm
/
DeepLabV3-Plus-MobileNet-Quantized

Image Segmentation
PyTorch
TF Lite
ONNX
quantized
android
Model card Files Community
qaihm-bot commited on
Commit
55cd912
•
1 Parent(s): 241c273

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +86 -30
README.md CHANGED
@@ -36,32 +36,28 @@ More details on model performance across various devices, can be found
36
 
37
  | Model | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
38
  |---|---|---|---|---|---|---|---|---|
39
- | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 3.296 ms | 0 - 1 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
40
- | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 5.17 ms | 0 - 17 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.so](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.so) |
41
- | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | ONNX | 4.232 ms | 11 - 18 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.onnx](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.onnx) |
42
- | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 2.457 ms | 0 - 65 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
43
- | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 3.855 ms | 1 - 31 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.so](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.so) |
44
- | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | ONNX | 3.086 ms | 0 - 70 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.onnx](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.onnx) |
45
- | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 2.287 ms | 0 - 41 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
46
- | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 3.837 ms | 1 - 26 MB | INT8 | NPU | Use Export Script |
47
- | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | ONNX | 2.529 ms | 0 - 47 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.onnx](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.onnx) |
48
- | DeepLabV3-Plus-MobileNet-Quantized | RB3 Gen 2 (Proxy) | QCS6490 Proxy | TFLITE | 14.219 ms | 5 - 48 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
49
- | DeepLabV3-Plus-MobileNet-Quantized | RB3 Gen 2 (Proxy) | QCS6490 Proxy | QNN | 17.715 ms | 1 - 9 MB | INT8 | NPU | Use Export Script |
50
- | DeepLabV3-Plus-MobileNet-Quantized | RB5 (Proxy) | QCS8250 Proxy | TFLITE | 119.151 ms | 11 - 23 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
51
- | DeepLabV3-Plus-MobileNet-Quantized | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 3.26 ms | 0 - 7 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
52
- | DeepLabV3-Plus-MobileNet-Quantized | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 3.948 ms | 1 - 2 MB | INT8 | NPU | Use Export Script |
53
- | DeepLabV3-Plus-MobileNet-Quantized | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 3.296 ms | 0 - 5 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
54
- | DeepLabV3-Plus-MobileNet-Quantized | SA8255 (Proxy) | SA8255P Proxy | QNN | 4.002 ms | 1 - 2 MB | INT8 | NPU | Use Export Script |
55
- | DeepLabV3-Plus-MobileNet-Quantized | SA8775 (Proxy) | SA8775P Proxy | TFLITE | 3.315 ms | 0 - 8 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
56
- | DeepLabV3-Plus-MobileNet-Quantized | SA8775 (Proxy) | SA8775P Proxy | QNN | 3.997 ms | 1 - 2 MB | INT8 | NPU | Use Export Script |
57
- | DeepLabV3-Plus-MobileNet-Quantized | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 3.345 ms | 0 - 2 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
58
- | DeepLabV3-Plus-MobileNet-Quantized | SA8650 (Proxy) | SA8650P Proxy | QNN | 3.98 ms | 1 - 2 MB | INT8 | NPU | Use Export Script |
59
- | DeepLabV3-Plus-MobileNet-Quantized | SA8295P ADP | SA8295P | TFLITE | 5.839 ms | 5 - 46 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
60
- | DeepLabV3-Plus-MobileNet-Quantized | SA8295P ADP | SA8295P | QNN | 6.876 ms | 1 - 6 MB | INT8 | NPU | Use Export Script |
61
- | DeepLabV3-Plus-MobileNet-Quantized | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 4.148 ms | 5 - 69 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
62
- | DeepLabV3-Plus-MobileNet-Quantized | QCS8450 (Proxy) | QCS8450 Proxy | QNN | 5.571 ms | 1 - 32 MB | INT8 | NPU | Use Export Script |
63
- | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 4.311 ms | 1 - 1 MB | INT8 | NPU | Use Export Script |
64
- | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon X Elite CRD | Snapdragon® X Elite | ONNX | 4.7 ms | 17 - 17 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.onnx](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.onnx) |
65
 
66
 
67
 
@@ -126,13 +122,73 @@ Profiling Results
126
  DeepLabV3-Plus-MobileNet-Quantized
127
  Device : Samsung Galaxy S23 (13)
128
  Runtime : TFLITE
129
- Estimated inference time (ms) : 3.3
130
- Estimated peak memory usage (MB): [0, 1]
131
- Total # Ops : 104
132
- Compute Unit(s) : NPU (104 ops)
133
  ```
134
 
135
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
136
 
137
 
138
  ## Run demo on a cloud-hosted device
 
36
 
37
  | Model | Device | Chipset | Target Runtime | Inference Time (ms) | Peak Memory Range (MB) | Precision | Primary Compute Unit | Target Model
38
  |---|---|---|---|---|---|---|---|---|
39
+ | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | TFLITE | 4.164 ms | 0 - 9 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
40
+ | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S23 | Snapdragon® 8 Gen 2 | QNN | 4.776 ms | 0 - 10 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.so](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.so) |
41
+ | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | TFLITE | 2.994 ms | 0 - 75 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
42
+ | DeepLabV3-Plus-MobileNet-Quantized | Samsung Galaxy S24 | Snapdragon® 8 Gen 3 | QNN | 3.442 ms | 1 - 36 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.so](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.so) |
43
+ | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | TFLITE | 2.801 ms | 0 - 51 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
44
+ | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon 8 Elite QRD | Snapdragon® 8 Elite | QNN | 3.45 ms | 1 - 31 MB | INT8 | NPU | Use Export Script |
45
+ | DeepLabV3-Plus-MobileNet-Quantized | RB3 Gen 2 (Proxy) | QCS6490 Proxy | TFLITE | 18.129 ms | 0 - 62 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
46
+ | DeepLabV3-Plus-MobileNet-Quantized | RB3 Gen 2 (Proxy) | QCS6490 Proxy | QNN | 19.952 ms | 1 - 9 MB | INT8 | NPU | Use Export Script |
47
+ | DeepLabV3-Plus-MobileNet-Quantized | RB5 (Proxy) | QCS8250 Proxy | TFLITE | 162.562 ms | 4 - 10 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
48
+ | DeepLabV3-Plus-MobileNet-Quantized | QCS8550 (Proxy) | QCS8550 Proxy | TFLITE | 4.143 ms | 0 - 3 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
49
+ | DeepLabV3-Plus-MobileNet-Quantized | QCS8550 (Proxy) | QCS8550 Proxy | QNN | 3.881 ms | 1 - 2 MB | INT8 | NPU | Use Export Script |
50
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8255 (Proxy) | SA8255P Proxy | TFLITE | 4.197 ms | 0 - 2 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
51
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8255 (Proxy) | SA8255P Proxy | QNN | 3.939 ms | 1 - 2 MB | INT8 | NPU | Use Export Script |
52
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8775 (Proxy) | SA8775P Proxy | TFLITE | 4.185 ms | 0 - 3 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
53
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8775 (Proxy) | SA8775P Proxy | QNN | 3.94 ms | 0 - 1 MB | INT8 | NPU | Use Export Script |
54
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8650 (Proxy) | SA8650P Proxy | TFLITE | 4.175 ms | 0 - 2 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
55
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8650 (Proxy) | SA8650P Proxy | QNN | 3.932 ms | 4 - 6 MB | INT8 | NPU | Use Export Script |
56
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8295P ADP | SA8295P | TFLITE | 6.632 ms | 0 - 50 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
57
+ | DeepLabV3-Plus-MobileNet-Quantized | SA8295P ADP | SA8295P | QNN | 7.203 ms | 1 - 6 MB | INT8 | NPU | Use Export Script |
58
+ | DeepLabV3-Plus-MobileNet-Quantized | QCS8450 (Proxy) | QCS8450 Proxy | TFLITE | 4.841 ms | 0 - 77 MB | INT8 | NPU | [DeepLabV3-Plus-MobileNet-Quantized.tflite](https://huggingface.co/qualcomm/DeepLabV3-Plus-MobileNet-Quantized/blob/main/DeepLabV3-Plus-MobileNet-Quantized.tflite) |
59
+ | DeepLabV3-Plus-MobileNet-Quantized | QCS8450 (Proxy) | QCS8450 Proxy | QNN | 5.666 ms | 1 - 34 MB | INT8 | NPU | Use Export Script |
60
+ | DeepLabV3-Plus-MobileNet-Quantized | Snapdragon X Elite CRD | Snapdragon® X Elite | QNN | 4.267 ms | 1 - 1 MB | INT8 | NPU | Use Export Script |
 
 
 
 
61
 
62
 
63
 
 
122
  DeepLabV3-Plus-MobileNet-Quantized
123
  Device : Samsung Galaxy S23 (13)
124
  Runtime : TFLITE
125
+ Estimated inference time (ms) : 4.2
126
+ Estimated peak memory usage (MB): [0, 9]
127
+ Total # Ops : 136
128
+ Compute Unit(s) : NPU (136 ops)
129
  ```
130
 
131
 
132
+ ## How does this work?
133
+
134
+ This [export script](https://aihub.qualcomm.com/models/deeplabv3_plus_mobilenet_quantized/qai_hub_models/models/DeepLabV3-Plus-MobileNet-Quantized/export.py)
135
+ leverages [Qualcomm® AI Hub](https://aihub.qualcomm.com/) to optimize, validate, and deploy this model
136
+ on-device. Lets go through each step below in detail:
137
+
138
+ Step 1: **Compile model for on-device deployment**
139
+
140
+ To compile a PyTorch model for on-device deployment, we first trace the model
141
+ in memory using the `jit.trace` and then call the `submit_compile_job` API.
142
+
143
+ ```python
144
+ import torch
145
+
146
+ import qai_hub as hub
147
+ from qai_hub_models.models.deeplabv3_plus_mobilenet_quantized import
148
+
149
+ # Load the model
150
+
151
+ # Device
152
+ device = hub.Device("Samsung Galaxy S23")
153
+
154
+
155
+ ```
156
+
157
+
158
+ Step 2: **Performance profiling on cloud-hosted device**
159
+
160
+ After compiling models from step 1. Models can be profiled model on-device using the
161
+ `target_model`. Note that this scripts runs the model on a device automatically
162
+ provisioned in the cloud. Once the job is submitted, you can navigate to a
163
+ provided job URL to view a variety of on-device performance metrics.
164
+ ```python
165
+ profile_job = hub.submit_profile_job(
166
+ model=target_model,
167
+ device=device,
168
+ )
169
+
170
+ ```
171
+
172
+ Step 3: **Verify on-device accuracy**
173
+
174
+ To verify the accuracy of the model on-device, you can run on-device inference
175
+ on sample input data on the same cloud hosted device.
176
+ ```python
177
+ input_data = torch_model.sample_inputs()
178
+ inference_job = hub.submit_inference_job(
179
+ model=target_model,
180
+ device=device,
181
+ inputs=input_data,
182
+ )
183
+ on_device_output = inference_job.download_output_data()
184
+
185
+ ```
186
+ With the output of the model, you can compute like PSNR, relative errors or
187
+ spot check the output with expected output.
188
+
189
+ **Note**: This on-device profiling and inference requires access to Qualcomm®
190
+ AI Hub. [Sign up for access](https://myaccount.qualcomm.com/signup).
191
+
192
 
193
 
194
  ## Run demo on a cloud-hosted device