Hugging Face's logo
Michael Goin's picture
88 17 19

Michael Goin

mgoin
markurtz's profile picture Parkerlambert123's profile picture heli-stand's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Recent Activity

published a model 1 day ago
RedHatAI/Phi-3-mini-128k-instruct-quantized.w8a16
published a model 1 day ago
RedHatAI/Phi-3-medium-128k-instruct-quantized.w8a16
published a model 1 day ago
RedHatAI/Phi-3-medium-128k-instruct-quantized.w8a8
View all activity

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Mistral AI_'s profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture Red Hat's profile picture

mgoin's activity

New activity in mistralai/Mistral-Small-3.1-24B-Instruct-2503 2 days ago

Address discrepancies in the languages supported by the Mistral Small 3.1 2503

1
#54 opened 5 days ago by
fpaupier
New activity in nm-testing/Mistral-Small-3.1-24B-Instruct-2503-FP8-dynamic 2 days ago

Please update the chat template

1
#1 opened 5 days ago by
stelterlab
New activity in mistralai/Mistral-Small-3.1-24B-Instruct-2503 2 days ago

FP8 Dynamic/W8A16 Quants Please

3
#44 opened 14 days ago by
rjmehta
New activity in mistralai/Mistral-Small-3.1-24B-Instruct-2503 7 days ago

Problem hosting the model using vllm

4
#45 opened 14 days ago by
ShaoServient
New activity in RedHatAI/Qwen2.5-VL-72B-Instruct-quantized.w8a8 about 1 month ago

Remove image_processor_type
#1 opened about 1 month ago by
pooya-davoodi-parasail
New activity in RedHatAI/Qwen2.5-VL-7B-Instruct-quantized.w8a8 about 1 month ago

Remove image_processor_type

1
#1 opened about 1 month ago by
pooya-davoodi-parasail
New activity in RedHatAI/Qwen2.5-VL-72B-Instruct-FP8-Dynamic about 1 month ago

Remove image_processor_type
#2 opened about 1 month ago by
pooya-davoodi-parasail
New activity in RedHatAI/Qwen2.5-VL-7B-Instruct-FP8-Dynamic about 1 month ago

Use Qwen2VLImageProcessor for image_processor_type

5
#2 opened about 2 months ago by
pooya-davoodi-parasail

Use Qwen2VLImageProcessor for image_processor_type
#3 opened about 2 months ago by
pooya-davoodi-parasail
New activity in cognitivecomputations/DeepSeek-R1-AWQ about 2 months ago

when i use vllm v0.7.2 to deploy r1 awq, i got empty content

13
#10 opened about 2 months ago by
bupalinyu

MLA is not supported with moe_wna16 quantization. Disabling MLA.

5
#7 opened about 2 months ago by
AMOSE
New activity in RedHatAI/gemma-2-9b-it-FP8 2 months ago

AttributeError: 'Gemma2Config' object has no attribute 'interleaved_sliding_window' Traceback (most recent call last):

2
#3 opened 2 months ago by
samos123
New activity in RedHatAI/granite-3.1-8b-instruct-FP8-dynamic 2 months ago

compressed-tensors MLA support requires fp8 activations and weights in group 'group_0',

2
#1 opened 2 months ago by
samos123
New activity in RedHatAI/Meta-Llama-3-8B-Instruct-FP8-KV 2 months ago

How to load this model?

2
#1 opened 9 months ago by
Frz614
New activity in RedHatAI/Llama-3.2-90B-Vision-Instruct-FP8-dynamic 4 months ago

Model does not run with VLLM

2
#3 opened 4 months ago by
aswad546
New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic 4 months ago

Nice model, any info on scripts used to quantize?

1
#1 opened 4 months ago by
RonanMcGovern
New activity in RedHatAI/Llama-3.2-11B-Vision-Instruct-FP8-dynamic 4 months ago

Thanks!
#2 opened 4 months ago by
Jindows
New activity in mistralai/Pixtral-Large-Instruct-2411 5 months ago

Add config_format and load_format to vLLM args
#5 opened 5 months ago by
mgoin

Update config.json to use null for sliding_window
#4 opened 5 months ago by
mgoin
New activity in mgoin/nemotron-3-8b-chat-4k-sft-hf 5 months ago

Adding `safetensors` variant of this model
#1 opened 5 months ago by
SFconvertbot