File size: 6,169 Bytes
500d737 d298905 500d737 d522d57 b7f84a9 0dc39b3 d298905 f89adb3 1699a28 f89adb3 0311e7b a7b16a0 b7f84a9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 |
---
license: creativeml-openrail-m
language:
- en
- de
- fr
- it
- pt
- hi
- es
- th
library_name: transformers
pipeline_tag: text-generation
tags:
- '1.0e-5'
- chain_of_thought
- ollama
- text-generation-inference
base_model:
- meta-llama/Llama-3.2-3B-Instruct
---
# **Llama-Thinker-3B-Preview**
Llama-Thinker-3B-Preview is a pretrained and instruction-tuned generative model designed for multilingual applications. These models are trained using synthetic datasets based on long chains of thought, enabling them to perform complex reasoning tasks effectively.
Model Architecture: [ Based on Llama 3.2 ] is an autoregressive language model that uses an optimized transformer architecture. The tuned versions undergo supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.
# **Use with transformers**
Starting with `transformers >= 4.43.0` onward, you can run conversational inference using the Transformers `pipeline` abstraction or by leveraging the Auto classes with the `generate()` function.
Make sure to update your transformers installation via `pip install --upgrade transformers`.
```python
import torch
from transformers import pipeline
model_id = "prithivMLmods/Llama-Thinker-3B-Preview"
pipe = pipeline(
"text-generation",
model=model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
{"role": "user", "content": "Who are you?"},
]
outputs = pipe(
messages,
max_new_tokens=256,
)
print(outputs[0]["generated_text"][-1])
```
Note: You can also find detailed recipes on how to use the model locally, with `torch.compile()`, assisted generations, quantised and more at [`huggingface-llama-recipes`](https://github.com/huggingface/huggingface-llama-recipes)
# **Use with `llama`**
Please, follow the instructions in the [repository](https://github.com/meta-llama/llama)
To download Original checkpoints, see the example command below leveraging `huggingface-cli`:
```
huggingface-cli download prithivMLmods/Llama-Thinker-3B-Preview --include "original/*" --local-dir Llama-Thinker-3B-Preview
```
Here’s a version tailored for the **Llama-Thinker-3B-Preview-GGUF** model:
---
# **How to Run Llama-Thinker-3B-Preview on Ollama Locally**
This guide demonstrates how to run the **Llama-Thinker-3B-Preview-GGUF** model locally using Ollama. The model is instruction-tuned for multilingual tasks and complex reasoning, making it highly versatile for a wide range of use cases. By the end, you'll be equipped to run this and other open-source models with ease.
---
## Example 1: How to Run the Llama-Thinker-3B-Preview Model
The **Llama-Thinker-3B** model is a pretrained and instruction-tuned LLM, designed for complex reasoning tasks across multiple languages. In this guide, we'll interact with it locally using Ollama, with support for quantized models.
### Step 1: Download the Model
First, download the **Llama-Thinker-3B-Preview-GGUF** model using the following command:
```bash
ollama run llama-thinker-3b-preview.gguf
```
### Step 2: Model Initialization and Download
Once the command is executed, Ollama will initialize and download the necessary model files. You should see output similar to this:
```plaintext
pulling manifest
pulling a12cd3456efg... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████▏ 3.2 GB
pulling 9f87ghijklmn... 100% ▕██████████████████████████████████████████████████████████████████████████████████████████▏ 6.5 KB
verifying sha256 digest
writing manifest
removing any unused layers
success
>>> Send a message (/? for help)
```
### Step 3: Interact with the Model
Once the model is fully loaded, you can interact with it by sending prompts. For example, let's ask:
```plaintext
>>> How can you assist me today?
```
A sample response might look like this [may / maynot be identical]:
```plaintext
I am Llama-Thinker-3B, an advanced AI language model designed to assist with complex reasoning, multilingual tasks, and general-purpose queries. Here are a few things I can help you with:
1. Answering complex questions in multiple languages.
2. Assisting with creative writing, content generation, and problem-solving.
3. Providing detailed summaries and explanations.
4. Translating text across different languages.
5. Generating ideas for personal or professional use.
6. Offering insights on technical topics.
Feel free to ask me anything you'd like assistance with!
```
### Step 4: Exit the Program
To exit the program, simply type:
```plaintext
/exit
```
---
## Example 2: Using Multi-Modal Models (Future Use)
In the future, Ollama may support multi-modal models where you can input both text and images for advanced interactions. This section will be updated as new capabilities become available.
---
## Notes on Using Quantized Models
Quantized models like **llama-thinker-3b-preview.gguf** are optimized for efficient performance on local systems with limited resources. Here are some key points to ensure smooth operation:
1. **VRAM/CPU Requirements**: Ensure your system has adequate VRAM or CPU resources to handle model inference.
2. **Model Format**: Use the `.gguf` model format for compatibility with Ollama.
---
# **Conclusion**
Running the **Llama-Thinker-3B-Preview** model locally using Ollama provides a powerful way to leverage open-source LLMs for complex reasoning and multilingual tasks. By following this guide, you can explore other models and expand your use cases as new models become available.
--- |