An Investigation of FP8 Across Accelerators for LLM Inference
Paper
•
2502.01070
•
Published
•
3
Régis Pierrard
regisss
AI & ML interests
None yet
Recent Activity
new activity
6 days ago
hf-doc-build/doc-build:add-version-executorch
liked
a Space
21 days ago
nanotron/ultrascale-playbook
posted
an
update
27 days ago
Nice paper comparing the fp8 inference efficiency of Nvidia H100 and Intel Gaudi2: https://huggingface.co/papers/2502.01070
The conclusion is interesting: "Our findings highlight that the Gaudi 2, by leveraging FP8, achieves higher throughput-to-power efficiency during LLM inference"
One aspect of AI hardware accelerators that is often overlooked is how they consume less energy than GPUs. It's nice to see researchers starting carrying out experiments to measure this!
Gaudi3 results soon...
Organizations
regisss's activity
upvoted
a
paper
28 days ago
upvoted
an
article
about 2 months ago
Article
Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference
•
71
upvoted
an
article
5 months ago
Article
Organizing a Privacy-preserving Hackathon
By
and 1 other
•
•
9
upvoted
an
article
10 months ago
Article
Energy Scores for AI Models
By
•
•
37