unsloth
/

DeepSeek-V3-GGUF

Inference Endpoints

Model card Files Files and versions Community

Resources

View closed (2)

Is it worth hosting a quantized DeepSeek V3? Cost & performance insights?

#15 opened 2 days ago by

Can you add 1.58bit?

#14 opened 4 days ago by

Dynamic quants

#13 opened 13 days ago by

Core dumped when merging DeepSeek-V3-Q2_K_XS

#12 opened 21 days ago by

First review, Q5-K-M require 502Gb RAM, better than Meta 405billions

#11 opened 29 days ago by

Issue with --n-gpu-layers 5 Parameter: Model Only Running on CPU

#10 opened 29 days ago by

I’m new to GGUF quants

#9 opened 30 days ago by

I loaded DeepSeek-V3-Q5_K_M up on my 10yrs old old Tesla M40 (Dell C4130)

#8 opened about 1 month ago by

why use q5 for key cache?

#7 opened about 1 month ago by

What is the required GPU size to run Is a 4090 possible and does it support ollama

#5 opened about 1 month ago by

I'm a newbie. How to use?

#4 opened about 1 month ago by

Getting error with Q3-K-M

#2 opened about 1 month ago by

Are these imatrix GGUF quants?

#1 opened about 1 month ago by