neuralmagic
/

Meta-Llama-3-8B-Instruct-FP8

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Meta-Llama-3-8B-Instruct-FP8 / README.md

mgoin's picture

Update README.md

2ae5589 verified 8 months ago

|

250 Bytes

metadata

tags:
  - fp8

Produced using https://github.com/neuralmagic/AutoFP8/blob/b0c1f789c51659bb023c06521ecbd04cea4a26f6/quantize.py

python quantize.py --model-id meta-llama/Meta-Llama-3-8B-Instruct --save-dir Meta-Llama-3-8B-Instruct-FP8