unsloth

/

DeepSeek-V3-GGUF

like 50

Follow

Unsloth AI 1.12k

Thanks for the quants first of everyone at Unloth but I was wondering if any of these were imatrix or if thre was any calibration done to get imatrix data to use on quants? If this is the case could we get the calibration data used? I was attempting this but ran into 24 hours of imartix calculation based on @bartowski 's great imatrix .txt here https://gist.github.com/auth/github?return_to=https%3A%2F%2Fgist.github.com%2Fbartowski1182%2Feb213dccb3571f863da82e99418f81e8

Hey kearm, no these are not imatrix - if it's a popular request we can definitely work on adding them! :)

Thanks for the quants first of everyone at Unloth but I was wondering if any of these were imatrix or if thre was any calibration done to get imatrix data to use on quants? If this is the case could we get the calibration data used? I was attempting this but ran into 24 hours of imartix calculation based on @bartowski 's great imatrix .txt here https://gist.github.com/auth/github?return_to=https%3A%2F%2Fgist.github.com%2Fbartowski1182%2Feb213dccb3571f863da82e99418f81e8

Hey kearm, no these are not imatrix - if it's a popular request we can definitely work on adding them! :)

@shimmyshimmer I was attempting to create them but the 24 hour calculation time on @bartowski 's imatrix data was too daunting given my machine is busy but given the historical prefromance of imatrix Q2_K_XS quants it seems nearly vital for closer to Q_4_K_M perforance. @bartowski Please weigh in if I am wrong.

@shimmyshimmer Just so we don't accidentally do the same thing twice I'm going to put a machine on making imatrix quants unless this is an already in process on your end.

@shimmyshimmer Just so we don't accidentally do the same thing twice I'm going to put a machine on making imatrix quants unless this is an already in process on your end.

Oh no we aren't doing it - you can definitely do it!