Edit Models filters

Inference status

Misc

compressed-tensors

Inference Endpoints

AutoTrain Compatible

text-generation-inference

8-bit precision

Misc with no match

4-bit precision

text-embeddings-inference

Carbon Emissions

Mixture of Experts

Models

557

Full-text search

Active filters: compressed-tensors

nm-testing/tinyllama-w4a16-compressed

Updated Oct 9, 2024 • 1.06k

nm-testing/tinyllama-w8a8-compressed

Updated Oct 9, 2024 • 1.15k

nm-testing/tinyllama-w8a16-dense

Updated Oct 9, 2024 • 1.06k

nm-testing/TinyLlama-1.1B-Chat-v1.0-actorder-weight-e2e

Text Generation • Updated about 22 hours ago • 301

nm-testing/TinyLlama-1.1B-Chat-v1.0-actorder-group

Text Generation • Updated Oct 9, 2024 • 2.96k

nm-testing/TinyLlama-1.1B-Chat-v1.0-actorder-weight

Text Generation • Updated Oct 9, 2024 • 13

nm-testing/TinyLlama-1.1B-Chat-v1.0-actorder-group-e2e

Text Generation • Updated about 22 hours ago • 428

nm-testing/tinyllama-fp8-dynamic-compressed-hf-quantizer

Updated Sep 11, 2024 • 1

nm-testing/tinyllama-w4a16-compressed-hf-quantizer

Updated Sep 11, 2024 • 13

nm-testing/tinyllama-w8a8-compressed-hf-quantizer

Updated Sep 11, 2024 • 27

nm-testing/tinyllama-w8a16-dense-hf-quantizer

Updated Sep 11, 2024 • 12

nm-testing/DeepSeek-Coder-V2-Lite-Instruct-W8A8

Updated Oct 9, 2024 • 9

nm-testing/Qwen2-VL-7B-Instruct-FP8-dynamic

Updated Oct 29, 2024 • 16

nm-testing/Phi-3.5-MoE-instruct-FP8

Updated Oct 9, 2024 • 14 • 2

nm-testing/OLMoE-1B-7B-0924-Instruct-FP8

Updated Oct 9, 2024 • 4

nm-testing/Meta-Llama-3.1-8B-Instruct-FP8-hf

Text Generation • Updated Sep 24, 2024 • 473

nm-testing/tinyllama-oneshot-w8a8-channel-dynamic-token-v2-asym

Updated Oct 9, 2024 • 2.29k

nm-testing/Meta-Llama-3-8B-Instruct-W8-Channel-A8-Dynamic-Asym-Per-Token-Test

Updated Oct 9, 2024 • 764 • 1

nm-testing/asym-w8w8-int8-static-per-tensor-tiny-llama

Updated Oct 9, 2024 • 2.3k

neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9, 2024 • 2.19k • 2

neuralmagic/Llama-3.2-3B-Instruct-FP8-dynamic

Text Generation • Updated Oct 9, 2024 • 1.46k • 2

neuralmagic/Llama-3.2-3B-Instruct-quantized.w8a8

Text Generation • Updated Oct 16, 2024 • 6.29k • 1

neuralmagic/Llama-3.2-1B-Instruct-FP8

Text Generation • Updated Oct 16, 2024 • 8.57k • 2

nm-testing/tiny-llama-w4a16-new-key

Updated Sep 26, 2024 • 2

neuralmagic/Llama-3.2-1B-quantized.w8a8

Updated Oct 9, 2024 • 1.09k

neuralmagic/Llama-3.2-3B-quantized.w8a8

Updated Oct 9, 2024 • 10

neuralmagic/Llama-3.2-1B-FP8

Updated Oct 9, 2024 • 65

neuralmagic/Phi-3.5-mini-instruct-FP8-KV

Text Generation • Updated Oct 1, 2024 • 175 • 2

horheynm/TinyLlama-1.1B-Chat-v1.0-QuantizedCache

Updated Sep 30, 2024 • 1

qeternity/Qwen2.5-72B-Instruct-FP8

Updated Oct 1, 2024 • 3