Spaces:
Running
on
CPU Upgrade
merged models marked as non merged in the leaderboard
Hi,
There are models in the leaderboard with false flags for merging, while the model description states clearly "merged" .. the submission to the leaderboard is "non_merged".. a fact that clearly constitutes a violation of the leaderboard classification terms.
Those guidelines and terms are set in place for clear reasons. Personally I find interesting some of those strategies, which highlights and exhibits relevant insights over neural networks.
So i guess we have two options here, either we remove the merged flag/selector.. or we enforce such fact. There is a big difference between running a script from two downloaded models and a training session with a full and long gradient flow :)
Hi @fblgit !
Do you have specific examples? We'll flag the models you report if we observe the same, as we agree there's a difference between simply merging and actually training a big model and spending the compute on that, hence why we had asked people to tag their merges initially.
https://huggingface.co/rombodawg/Rombos-LLM-V2.5-Qwen-32b/discussions/2
But there are more like this in the TOP50, but Im not competing on 70/72B range for now. Also wanted to add that the model is not contaminated, so FLAG im not sure its the right thing.. its mostly about setting correctly the merged flag and thats it to be fair.
Missed your answer, sorry! We used to flag models which do not correctly report the merge tag to avoid misdirecting users - if you take some time to setup a list I'll add the flags