File size: 2,883 Bytes
99f83af 7e08cd6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
---
license: mit
---
# DOLG in torch and tensorflow (TF2)
Re-implementation (Non Official) of the paper DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features accepted at ICCV 2021.
[paper](https://arxiv.org/pdf/2108.02927.pdf)
The pytorch checkpoint has been converted into tensorflow format (.h5) from this repository : https://github.com/feymanpriv/DOLG (Official)
## Installation
> pip install opencv-python==4.5.5.64
> pip install huggingface-hub
to install dolg :
> pip install dolg
OR
> pip install -e .
## Inference
To do some inference on single sample, you can use python script in examples/ folder or use as follows:
```
import dolg
import numpy as np
from dolg.utils.extraction import process_data
depth = 50
# for pytorch
import torch
from dolg.dolg_model_pt import DOLG
from dolg.resnet_pt import ResNet
backbone = ResNet(depth=depth, num_groups=1, width_per_group=64, bn_eps=1e-5,
bn_mom=0.1, trans_fun="bottleneck_transform")
model = DOLG(backbone, s4_dim=2048, s3_dim=1024, s2_dim=512, head_reduction_dim=512,
with_ma=False, num_classes=None, pretrained=f"r{depth}")
img = process_data("image.jpg", "", mode="pt").unsqueeze(0)
with torch.no_grad():
output = model(img)
print(output)
# for tensorflow
import tensorflow as tf
from dolg.dolg_model_tf2 import DOLG
from dolg.resnet_tf2 import ResNet
backbone = ResNet(depth=depth, num_groups=1, width_per_group=64, bn_eps=1e-5,
bn_mom=0.1, trans_fun="bottleneck_transform", name="globalmodel")
model = DOLG(backbone, s4_dim=2048, s3_dim=1024, s2_dim=512, head_reduction_dim=512,
with_ma=False, num_classes=None, pretrained=f"r{depth}")
img = process_data("image.jpg", "", mode="tf")
img = np.expand_dims(img, axis=0)
output = model.predict(img)
print(output)
```
## Data
The model has been trained on google landmark v2. You can find the dataset on the official repository : https://github.com/cvdfoundation/google-landmark .
# Citation :
```bibtex
@misc{yang2021dolg,
title={DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features},
author={Min Yang and Dongliang He and Miao Fan and Baorong Shi and Xuetong Xue and Fu Li and Errui Ding and Jizhou Huang},
year={2021},
eprint={2108.02927},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@misc{https://doi.org/10.48550/arxiv.2004.01804,
doi = {10.48550/ARXIV.2004.01804},
url = {https://arxiv.org/abs/2004.01804},
author = {Weyand, Tobias and Araujo, Andre and Cao, Bingyi and Sim, Jack},
keywords = {Computer Vision and Pattern Recognition (cs.CV), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval},
``` |