Edit Models filters

Inference status

Misc

compressed-tensors

Inference Endpoints

AutoTrain Compatible

text-generation-inference

8-bit precision

Misc with no match

4-bit precision

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

557

Full-text search

Active filters: compressed-tensors

neuralmagic/gemma-2-2b-it-quantized.w8a16

Text Generation • Updated Oct 9, 2024 • 10 • 1

neuralmagic/gemma-2-2b-quantized.w8a16

Text Generation • Updated Oct 9, 2024 • 58

nm-testing/Mixtral-8x7B-Instruct-v0.1-W4A16-channel-quantized

Updated Oct 9, 2024 • 121

neuralmagic/gemma-2-9b-it-quantized.w4a16

Text Generation • Updated Oct 9, 2024 • 218 • 1

neuralmagic/gemma-2-2b-it-quantized.w4a16

Text Generation • Updated Oct 9, 2024 • 857 • 1

neuralmagic/SmolLM-1.7B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9, 2024 • 31

neuralmagic/Mistral-Nemo-Instruct-2407-quantized.w4a16

Text Generation • Updated Oct 9, 2024 • 2.32k • 3

neuralmagic/gemma-2-2b-it-quantized.w8a8

Text Generation • Updated Oct 9, 2024 • 2.19k

neuralmagic/gemma-2-9b-it-quantized.w8a8

Text Generation • Updated Oct 9, 2024 • 301 • 2

neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • Updated Dec 3, 2024 • 393 • 2

neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • Updated Oct 9, 2024 • 194 • 2

nm-testing/Meta-Llama-3-8B-Instruct-fp8-hf_compat

Updated Aug 30, 2024 • 19

neuralmagic/SmolLM-360M-Instruct-quantized.w8a8

Text Generation • Updated Oct 9, 2024 • 34

neuralmagic/SmolLM-135M-Instruct-quantized.w8a8

Text Generation • Updated Oct 9, 2024 • 221

nm-testing/SmolLM-135M-Instruct-quantized.w4a16

Text Generation • Updated Oct 9, 2024 • 6

nm-testing/SmolLM-360M-Instruct-quantized.w4a16

Updated Oct 9, 2024

nm-testing/SmolLM-1.7B-Instruct-quantized.w4a16

Text Generation • Updated Oct 9, 2024 • 5

neuralmagic/SmolLM-1.7B-Instruct-quantized.w8a8

Text Generation • Updated Oct 9, 2024 • 17 • 1

nm-testing/TinyLlama-1.1B-Chat-v1.0-W4A16_channel-e2e

Text Generation • Updated about 22 hours ago • 279

nm-testing/TinyLlama-1.1B-Chat-v1.0-W4A16-e2e

Text Generation • Updated about 22 hours ago • 300

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A16_channel-e2e

Text Generation • Updated about 22 hours ago • 336

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A16-e2e

Text Generation • Updated about 22 hours ago • 348

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8A16_channel-e2e

Text Generation • Updated about 23 hours ago • 55

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8A16_tensor-e2e

Text Generation • Updated about 23 hours ago • 20

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8-e2e

Text Generation • Updated about 23 hours ago • 372

nm-testing/TinyLlama-1.1B-Chat-v1.0-FP8_DYNAMIC-e2e

Text Generation • Updated about 23 hours ago • 58

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A8-e2e

Text Generation • Updated about 23 hours ago • 37

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A8_tensor_weight_static_per_tensor_act-e2e

Text Generation • Updated about 23 hours ago • 380

nm-testing/TinyLlama-1.1B-Chat-v1.0-W8A8_channel_weight_static_per_tensor-e2e

Text Generation • Updated about 23 hours ago • 331

nm-testing/tinyllama-fp8-dynamic-compressed

Updated Oct 9, 2024 • 1.08k