|
--- |
|
language: en |
|
tags: |
|
- llama |
|
- quantization |
|
- text-generation |
|
- cpu |
|
- gpu |
|
- efficient-inference |
|
license: apache-2.0 |
|
base_model: |
|
- meta-llama/Llama-3.1-8B |
|
--- |
|
|
|
# Llama 3.1 8B Q4_K_M GGUF Model |
|
|
|
## Overview |
|
This is the quantized version of the Llama 3.1 8B model in Q4_K_M format, optimized for efficient inference on both CPU and GPU. The model was quantized using the llama.cpp library, allowing users to run it in resource-constrained environments . This quantization reduces the model's memory footprint while maintaining strong language generation capabilities. |
|
|
|
The model was originally trained by Meta AI and has been adapted to the GGUF format for compatibility with llama.cpp. |
|
|
|
## Model Details |
|
- **Base Model**: meta-llama/Llama-3.1-8B |
|
- **Quantization Type**: Q4_K_M (4-bit quantization with memory optimization) |
|
- **Model Size**: 8B parameters |
|
- **Format**: GGUF (used for efficient loading in llama.cpp) |
|
- **Intended Use**: Text generation, inference on CPUs/GPUs with reduced memory constraints |
|
|
|
## Intended Use |
|
The model is intended for text generation tasks and is optimized for efficient inference on both CPUs and GPUs, making it suitable for use in resource-constrained environments. |
|
|
|
## License |
|
This model is licensed under the Apache 2.0 License. |
|
|
|
--- |
