timm

/

mobilenetv4_conv_small.e2400_r224_in1k

like 7

Hi @Elsospi ,
I agree with your assessment.
@rwightman ?
Best,
M

@Elsospi @mikehemberger it depends what tradeoffs you want, higher accuracy or faster. The train/test res discrepeancy when training w/ augmentations that have lots of crop/zoom typically gives you a higher accuracy when you test at a higher resolution than train. As I train new models I often run val at a few res and find a good balance to put in the 'test' defaults and run my benchmarks at both train & test res.

For downstream use, you can try both, see what works best. If you fine-tune you might 'erase' some of that benefit with low-aug fine-tune and find it makes not much difference. The timm scripts havea --use-train-size argument to ignore the test size & crop for eval.

In the recent leaderboard I added, I made sure I have numbers for all model arch + img size tuples to cover both train & test for all models that have both. This is new but useful, you can compare arch and see the throughputs and accuracies for all arcjh+img_size tuples. On some of the OOD evals you'll see there's less benefit of the train/test discrep.

Specifically for these recent mnv4 models, the rxxx value in the weight tag is the train res. And in the summary in the model cards for the mnodel results you'll see an img_size for both that value and a test size.

https://huggingface.co/spaces/timm/leaderboard

Thank you @rwightman !
That sounds like a neat approach to introduce some small variation / randomness to the pipeline. Like it a lot!
Best,
Mike