ruslanmv commited on
Commit
2161ae5
·
verified ·
1 Parent(s): cfb59eb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +74 -6
README.md CHANGED
@@ -2,6 +2,7 @@
2
  base_model: unsloth/meta-llama-3.1-8b-bnb-4bit
3
  language:
4
  - en
 
5
  license: apache-2.0
6
  tags:
7
  - text-generation-inference
@@ -12,12 +13,79 @@ tags:
12
  - sft
13
  ---
14
 
15
- # Uploaded model
16
 
17
- - **Developed by:** ruslanmv
18
- - **License:** apache-2.0
19
- - **Finetuned from model :** unsloth/meta-llama-3.1-8b-bnb-4bit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
22
 
23
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
2
  base_model: unsloth/meta-llama-3.1-8b-bnb-4bit
3
  language:
4
  - en
5
+ - it
6
  license: apache-2.0
7
  tags:
8
  - text-generation-inference
 
13
  - sft
14
  ---
15
 
16
+ # Meta LLaMA 3.1 8B BNB 4-bit Finetuned Model
17
 
18
+ This model is a fine-tuned version of `unsloth/meta-llama-3.1-8b-bnb-4bit`, developed by **ruslanmv** for text generation tasks. It leverages 4-bit quantization, making it more efficient for inference while maintaining strong performance in natural language generation.
19
+
20
+ ---
21
+
22
+ ## Model Details
23
+
24
+ - **Base Model**: `unsloth/meta-llama-3.1-8b-bnb-4bit`
25
+ - **Finetuned by**: ruslanmv
26
+ - **Language**: English
27
+ - **License**: [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
28
+ - **Tags**:
29
+ - text-generation-inference
30
+ - transformers
31
+ - unsloth
32
+ - llama
33
+ - trl
34
+ - sft
35
+
36
+ ---
37
+
38
+ ## Model Usage
39
+
40
+ ### Installation
41
+
42
+ To use this model, you will need to install the necessary libraries:
43
+
44
+ ```bash
45
+ pip install transformers accelerate bitsandbytes
46
+ ```
47
+
48
+ ### Loading the Model in Python
49
+
50
+ Here’s an example of how to load this fine-tuned model using Hugging Face's `transformers` library:
51
+
52
+ ```python
53
+ from transformers import AutoModelForCausalLM, AutoTokenizer
54
+ import torch
55
+
56
+ # Load the model and tokenizer
57
+ model_name = "ruslanmv/meta-llama-3.1-8b-bnb-4bit"
58
+
59
+ # Ensure you have the right device setup
60
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
61
+
62
+ # Load the model and tokenizer from the Hugging Face Hub
63
+ model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", torch_dtype=torch.float16)
64
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
65
+
66
+ # Example usage
67
+ input_text = "Recupera il conteggio di tutte le righe nella tabella table1"
68
+ inputs = tokenizer(input_text, return_tensors="pt").to(device)
69
+
70
+ # Generate output text
71
+ outputs = model.generate(**inputs, max_length=50)
72
+
73
+ # Decode and print the generated text
74
+ generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
75
+ print(generated_text)
76
+ ```
77
+
78
+ ### Model Features
79
+
80
+ - **Text Generation**: This model is fine-tuned to generate coherent and contextually accurate text based on the provided input.
81
+ - **Efficiency**: Using 4-bit quantization with the `bitsandbytes` library, it optimizes memory and inference performance.
82
+
83
+ ### License
84
+
85
+ This model is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). You are free to use, modify, and distribute this model, provided that you comply with the license terms.
86
+
87
+ ### Acknowledgments
88
+
89
+ This model was fine-tuned by **ruslanmv** based on the original work of `unsloth` and the `meta-llama-3.1-8b-bnb-4bit` model.
90
 
 
91