Ross Wightman

FWIW, I just found a bug in the pipeline, it's applying sigmoid instead of softmax by default due to a bug (https://github.com/huggingface/transformers/pull/35848), so add function_to_apply='softmax' if you want softmax probs... this is not specific to the timm integration and looks like it's been there a while. I did confirm that if you set the labels as above the fine-tuned timm model will predict with the correct labels and should push to hub with those as well...

@davidrs this part in the example to reset classifier / labels with the target dataset is important

model = AutoModelForImageClassification.from_pretrained(
    checkpoint,
    num_labels=num_labels,
    id2label=id2label,
    label2id=label2id,
    ignore_mismatched_sizes=True,
)

If you don't do that I believe it gets pushed with the original labels, an imagenet-1k (1000) class classifier, etc. Though I guess the lowest 'n' classes that are in the new dataset would be fine-tuned with the target (if it's less, it'd crash if the new dataset has more classes)

@davidrs hmm, possible there's an issue with the label handling, there was a change made to the integration near release to keep the labels compatible with timm use (keep label_names field instead of id2label/label2id), there's actually a mix of the two cases and many of @ariG23498 fine-tunes have the id2label, though I was told at release time it should be producing label_names...

Do you have a public model I could look at? You're using the example image classification script in Transformers (https://github.com/huggingface/transformers/tree/main/examples/pytorch/image-classification) or using Trainer directly yourself in a custom script/notebook as in the example above?