neuralmagic
/

Meta-Llama-3-8B-Instruct-FP8

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Meta-Llama-3-8B-Instruct-FP8 / README.md

mgoin's picture

Create README.md

ee0911f verified 8 months ago

|

230 Bytes

Produced using https://github.com/neuralmagic/AutoFP8/blob/b0c1f789c51659bb023c06521ecbd04cea4a26f6/quantize.py

python quantize.py --model-id meta-llama/Meta-Llama-3-8B-Instruct --save-dir Meta-Llama-3-8B-Instruct-FP8