File size: 3,262 Bytes
c02be22 2161ae5 c02be22 cfb59eb c02be22 944c3a5 c02be22 944c3a5 2161ae5 32e2418 2161ae5 944c3a5 2161ae5 6af5ebe 2161ae5 6af5ebe 2161ae5 6af5ebe 2161ae5 32e2418 2161ae5 c02be22 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 |
---
language:
- en
- it
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
---
# Meta LLaMA 3.1 8B 4-bit Finetuned Model
This model is a fine-tuned version of `Meta-Llama-3.1-8B`, developed by **ruslanmv** for text generation tasks. It leverages 4-bit quantization, making it more efficient for inference while maintaining strong performance in natural language generation.
---
## Model Details
- **Base Model**: `unsloth/meta-llama-3.1-8b-bnb-4bit`
- **Finetuned by**: ruslanmv
- **Language**: English
- **License**: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
- **Tags**:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
- sft
---
## Model Usage
### Installation
To use this model, you will need to install the necessary libraries:
```bash
pip install transformers accelerate
```
### Loading the Model in Python
Here’s an example of how to load this fine-tuned model using Hugging Face's `transformers` library:
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
# Load the model and tokenizer
model_name = "ruslanmv/Meta-Llama-3.1-8B-Text-to-SQL"
# Ensure you have the right device setup
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# Load the model and tokenizer from the Hugging Face Hub
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
tokenizer = AutoTokenizer.from_pretrained(model_name)
# Initialize the tokenizer (adjust the model name as needed)
# Define EOS token for terminating the sequences
EOS_TOKEN = tokenizer.eos_token
# Define Alpaca-style prompt template
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{}
### Input:
{}
### Response:
"""
# Format the prompt without the response part
prompt = alpaca_prompt.format(
"Provide the SQL query",
"Seleziona tutte le colonne della tabella table1 dove la colonna anni è uguale a 2020"
)
# Tokenize the prompt and generate text
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=64, use_cache=True)
# Decode the generated text
generated_text = tokenizer.batch_decode(outputs, skip_special_tokens=True)[0]
# Extract the generated response only (remove the prompt part)
response_start = generated_text.find("### Response:") + len("### Response:\n")
response = generated_text[response_start:].strip()
# Print the response (excluding the prompt)
print(response)
```
and the answer is
```
SELECT * FROM table1 WHERE anni = 2020
```
### Model Features
- **Text Generation**: This model is fine-tuned to generate coherent and contextually accurate text based on the provided input.
### License
This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). You are free to use, modify, and distribute this model, provided that you comply with the license terms.
### Acknowledgments
This model was fine-tuned by **ruslanmv** based on the original work of `unsloth` and the `meta-llama-3.1-8b-bnb-4bit` model.
|