Post
420
Why the 'how many r's in strawberry' prompt "breaks" llama4? :D
Quants DevQuasar/meta-llama.Llama-4-Scout-17B-16E-Instruct-GGUF
Quants DevQuasar/meta-llama.Llama-4-Scout-17B-16E-Instruct-GGUF
Csaba Kecskemeti PRO
csabakecskemeti
AI & ML interests
None yet
Recent Activity
updated
a model
11 minutes ago
DevQuasar/meta-llama.Llama-4-Scout-17B-16E-Instruct-GGUF
updated
a model
about 4 hours ago
DevQuasar/open-thoughts.OpenThinker2-7B-GGUF
published
a model
about 5 hours ago
DevQuasar/open-thoughts.OpenThinker2-7B-GGUF
Organizations
Posts
25
Post
3350
I'm collecting llama-bench results for inference with a llama 3.1 8B q4 and q8 reference models on varoius GPUs. The results are average of 5 executions.
The system varies (different motherboard and CPU ... but that probably that has little effect on the inference performance).
https://devquasar.com/gpu-gguf-inference-comparison/
the exact models user are in the page
I'd welcome results from other GPUs is you have access do anything else you've need in the post. Hopefully this is useful information everyone.
The system varies (different motherboard and CPU ... but that probably that has little effect on the inference performance).
https://devquasar.com/gpu-gguf-inference-comparison/
the exact models user are in the page
I'd welcome results from other GPUs is you have access do anything else you've need in the post. Hopefully this is useful information everyone.
datasets
None public yet