|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- eugenesiow/Div2k |
|
- eugenesiow/Set5 |
|
language: |
|
- en |
|
tags: |
|
- RyzenAI |
|
- super resolution |
|
- SISR |
|
- pytorch |
|
--- |
|
## Model description |
|
SESR is based on linear overparameterization of CNNs and creates an efficient model architecture for SISR. It was introduced in the paper [Collapsible Linear Blocks for Super-Efficient Super Resolution](https://arxiv.org/abs/2103.09404). |
|
The official code for this work is available at this |
|
https://github.com/ARM-software/sesr |
|
|
|
We develop a modified version that could be supported by [AMD Ryzen AI](https://onnxruntime.ai/docs/execution-providers/Vitis-AI-ExecutionProvider.html). |
|
|
|
## Intended uses & limitations |
|
|
|
You can use the raw model for super resolution. See the [model hub](https://huggingface.co/models?search=amd/sesr) to look for all available models. |
|
|
|
|
|
## How to use |
|
|
|
### Installation |
|
|
|
Follow [Ryzen AI Installation](https://ryzenai.docs.amd.com/en/latest/inst.html) to prepare the environment for Ryzen AI. |
|
Run the following script to install pre-requisites for this model. |
|
```bash |
|
pip install -r requirements.txt |
|
``` |
|
|
|
|
|
### Data Preparation (optional: for accuracy evaluation) |
|
|
|
1. Download the benchmark(https://cv.snu.ac.kr/research/EDSR/benchmark.tar) dataset. |
|
2. Organize the dataset directory as follows: |
|
```Plain |
|
└── dataset |
|
└── benchmark |
|
├── Set5 |
|
├── HR |
|
| ├── baby.png |
|
| ├── ... |
|
└── LR_bicubic |
|
└──X2 |
|
├──babyx2.png |
|
├── ... |
|
├── Set14 |
|
├── ... |
|
``` |
|
|
|
### Test & Evaluation |
|
|
|
- Code snippet from [`one_image_inference.py`](one_image_inference.py) on how to use |
|
```python |
|
parser = argparse.ArgumentParser(description='EDSR and MDSR') |
|
parser.add_argument('--onnx_path', type=str, default='SESR_int8.onnx', |
|
help='onnx path') |
|
parser.add_argument('--image_path', default='test_data/test.png', |
|
help='path of your image') |
|
parser.add_argument('--output_path', default='test_data/sr.png', |
|
help='path of your image') |
|
parser.add_argument('--ipu', action='store_true', |
|
help='use ipu') |
|
parser.add_argument('--provider_config', type=str, default=None, |
|
help='provider config path') |
|
args = parser.parse_args() |
|
if args.ipu: |
|
providers = ["VitisAIExecutionProvider"] |
|
provider_options = [{"config_file": args.provider_config}] |
|
else: |
|
providers = ['CUDAExecutionProvider', 'CPUExecutionProvider'] |
|
provider_options = None |
|
|
|
onnx_file_name = args.onnx_path |
|
image_path = args.image_path |
|
output_path = args.output_path |
|
|
|
ort_session = onnxruntime.InferenceSession(onnx_file_name, providers=providers, provider_options=provider_options) |
|
lr = cv2.imread(image_path)[np.newaxis,:,:,:].transpose((0,3,1,2)).astype(np.float32) |
|
sr = tiling_inference(ort_session, lr, 8, (56, 56)) |
|
sr = np.clip(sr, 0, 255) |
|
sr = sr.squeeze().transpose((1,2,0)).astype(np.uint8) |
|
sr = cv2.imwrite(output_path, sr) |
|
``` |
|
|
|
- Run inference for a single image |
|
```python |
|
python one_image_inference.py --onnx_path SESR_int8.onnx --image_path /Path/To/Your/Image --ipu --provider_config Path/To/vaip_config.json |
|
``` |
|
Note: **vaip_config.json** is located at the setup package of Ryzen AI (refer to [Installation](https://huggingface.co/amd/yolox-s#installation)) |
|
|
|
- Test accuracy of the quantized model |
|
```python |
|
python test.py --onnx_path SESR_int8.onnx --data_test Set5 --ipu --provider_config Path/To/vaip_config.json |
|
``` |
|
|
|
|
|
|
|
### Performance |
|
| Method | Scale | Flops | Set5 | |
|
|------------|-------|-------|--------------| |
|
|SESR-S (float) |X2 |10.22G |37.21| |
|
|SESR-S (INT8) |X2 |10.22G |36.81| |
|
- Note: the Flops is calculated with the input resolution is 256x256 |
|
|
|
|
|
```bibtex |
|
@misc{bhardwaj2022collapsible, |
|
title={Collapsible Linear Blocks for Super-Efficient Super Resolution}, |
|
author={Kartikeya Bhardwaj and Milos Milosavljevic and Liam O'Neil and Dibakar Gope and Ramon Matas and Alex Chalfin and Naveen Suda and Lingchuan Meng and Danny Loh}, |
|
year={2022}, |
|
eprint={2103.09404}, |
|
archivePrefix={arXiv}, |
|
primaryClass={eess.IV} |
|
} |
|
``` |
