Hugging Face's logo

mgoin
/
Meta-Llama-3-70B-Instruct-Marlin

Text Generation
Transformers
PyTorch
English
llama
facebook
meta
llama-3
conversational
Inference Endpoints
4-bit precision
gptq
Model card Files Community
2

What conversion process are you using?

#2
by matt-psaltis-devbricks - opened
Discussion
matt-psaltis-devbricks

Curious which approach you're using to convert these models to Marlin? is it a published process?

Cheers!

mgoin
 Owner

Hi @matt-psaltis-devbricks yep I used this script https://github.com/robertgshaw2-neuralmagic/marlin-example/blob/master/apply_gptq_save_marlin.py

matt-psaltis-devbricks

Awesome! Cheers for that

matt-psaltis-devbricks changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment