File size: 5,914 Bytes
84de7c2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9fce119
84de7c2
9fce119
 
84de7c2
9fce119
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84de7c2
 
 
 
 
9fce119
 
84de7c2
 
 
9fce119
84de7c2
 
9fce119
 
84de7c2
9fce119
84de7c2
 
 
 
 
 
9fce119
84de7c2
9fce119
 
 
84de7c2
9fce119
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
---
license: apache-2.0
base_model: nidum/Nidum-Llama-3.2-3B-Uncensored
library_name: adapter-transformers
tags:
- chemistry
- biology
- legal
- code
- medical
- finance
- mlx
pipeline_tag: text-generation
---

### Nidum-Llama-3.2-3B-Uncensored-MLX-4bit  

### Welcome to Nidum!  
At Nidum, we are committed to delivering cutting-edge AI models that offer advanced capabilities and unrestricted access to innovation. With **Nidum-Llama-3.2-3B-Uncensored-MLX-4bit**, we bring you a performance-optimized, space-efficient, and feature-rich model designed for diverse use cases.  

---

[![GitHub Icon](https://upload.wikimedia.org/wikipedia/commons/thumb/9/95/Font_Awesome_5_brands_github.svg/232px-Font_Awesome_5_brands_github.svg.png)](https://github.com/NidumAI-Inc)  
**Explore Nidum's Open-Source Projects on GitHub**: [https://github.com/NidumAI-Inc](https://github.com/NidumAI-Inc)

---

### Key Features

1. **Compact and Efficient**: Built in the **MLX-4bit format** for optimized performance with minimal memory usage.  
2. **Versatility**: Excels in a wide range of tasks, including technical problem-solving, educational queries, and casual conversations.  
3. **Extended Context Handling**: Capable of maintaining coherence in long-context interactions.  
4. **Seamless Integration**: Enhanced compatibility with the **mlx-lm library** for a streamlined development experience.  
5. **Uncensored Access**: Provides uninhibited responses across a variety of topics and applications.  

---

### How to Use

To utilize **Nidum-Llama-3.2-3B-Uncensored-MLX-4bit**, install the **mlx-lm** library and follow the example code below:

#### Installation

```bash
pip install mlx-lm
```

#### Usage

```python
from mlx_lm import load, generate

# Load the model and tokenizer
model, tokenizer = load("nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-4bit")

# Create a prompt
prompt = "hello"

# Apply the chat template if available
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )

# Generate the response
response = generate(model, tokenizer, prompt=prompt, verbose=True)

# Print the response
print(response)
```

---

### About the Model

The **nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-4bit** model was converted to MLX format from **nidum/Nidum-Llama-3.2-3B-Uncensored** using **mlx-lm version 0.19.2**, providing the following benefits:  

- **Smaller Memory Footprint**: Ideal for environments with limited hardware resources.  
- **High Performance**: Retains the advanced capabilities of the original model while optimizing inference speed and efficiency.  
- **Plug-and-Play Compatibility**: Easily integrate with the **mlx-lm** ecosystem for seamless deployment.  

---

### Use Cases

- **Technical Problem Solving**  
- **Research and Educational Assistance**  
- **Open-Ended Q&A**  
- **Creative Writing and Ideation**  
- **Long-Context Dialogues**  
- **Unrestricted Knowledge Exploration**  

---

### Datasets and Fine-Tuning  

The model inherits the fine-tuned capabilities of its predecessor, **Nidum-Llama-3.2-3B-Uncensored**, including:  

- **Uncensored Data**: Ensures detailed and uninhibited responses.  
- **RAG-Based Fine-Tuning**: Optimizes retrieval-augmented generation for information-intensive tasks.  
- **Math-Instruct Data**: Tailored for precise mathematical reasoning.  
- **Long-Context Fine-Tuning**: Enhanced coherence and relevance in extended interactions.  

---

### Quantized Model Download  

The **MLX-4bit** version is highly efficient, maintaining a balance between precision and memory usage.  

---

#### Benchmark

| **Benchmark**    | **Metric**                       | **LLaMA 3B** | **Nidum 3B** | **Observation**                                                                                     |
|-------------------|-----------------------------------|--------------|--------------|-----------------------------------------------------------------------------------------------------|
| **GPQA**         | Exact Match (Flexible)           | 0.3          | 0.5          | Nidum 3B demonstrates significant improvement, particularly in **generative tasks**.                |
|                  | Accuracy                         | 0.4          | 0.5          | Consistent improvement, especially in **zero-shot** scenarios.                                      |
| **HellaSwag**    | Accuracy                         | 0.3          | 0.4          | Better performance in **common sense reasoning** tasks.                                             |
|                  | Normalized Accuracy              | 0.3          | 0.4          | Enhanced ability to understand and predict context in sentence completion.                          |
|                  | Normalized Accuracy (Stderr)     | 0.15275      | 0.1633       | Slightly improved consistency in normalized accuracy.                                               |
|                  | Accuracy (Stderr)                | 0.15275      | 0.1633       | Shows robustness in reasoning accuracy compared to LLaMA 3B.                                        |

---

### Insights:

1. **Compact Efficiency**: The MLX-4bit model ensures high performance with reduced resource usage.  
2. **Enhanced Usability**: Optimized for seamless integration with lightweight deployment scenarios.  

---

### Contributing  

We invite contributions to further enhance the **MLX-4bit** model's capabilities. Reach out to us for collaboration opportunities.  

---

### Contact  

For inquiries, support, or feedback, email us at **[email protected]**.  

---

### Explore the Future  

Embrace the power of innovation with **Nidum-Llama-3.2-3B-Uncensored-MLX-4bit**—the ideal blend of performance and efficiency.  

---