File size: 5,914 Bytes
84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 84de7c2 9fce119 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 |
---
license: apache-2.0
base_model: nidum/Nidum-Llama-3.2-3B-Uncensored
library_name: adapter-transformers
tags:
- chemistry
- biology
- legal
- code
- medical
- finance
- mlx
pipeline_tag: text-generation
---
### Nidum-Llama-3.2-3B-Uncensored-MLX-4bit
### Welcome to Nidum!
At Nidum, we are committed to delivering cutting-edge AI models that offer advanced capabilities and unrestricted access to innovation. With **Nidum-Llama-3.2-3B-Uncensored-MLX-4bit**, we bring you a performance-optimized, space-efficient, and feature-rich model designed for diverse use cases.
---
[![GitHub Icon](https://upload.wikimedia.org/wikipedia/commons/thumb/9/95/Font_Awesome_5_brands_github.svg/232px-Font_Awesome_5_brands_github.svg.png)](https://github.com/NidumAI-Inc)
**Explore Nidum's Open-Source Projects on GitHub**: [https://github.com/NidumAI-Inc](https://github.com/NidumAI-Inc)
---
### Key Features
1. **Compact and Efficient**: Built in the **MLX-4bit format** for optimized performance with minimal memory usage.
2. **Versatility**: Excels in a wide range of tasks, including technical problem-solving, educational queries, and casual conversations.
3. **Extended Context Handling**: Capable of maintaining coherence in long-context interactions.
4. **Seamless Integration**: Enhanced compatibility with the **mlx-lm library** for a streamlined development experience.
5. **Uncensored Access**: Provides uninhibited responses across a variety of topics and applications.
---
### How to Use
To utilize **Nidum-Llama-3.2-3B-Uncensored-MLX-4bit**, install the **mlx-lm** library and follow the example code below:
#### Installation
```bash
pip install mlx-lm
```
#### Usage
```python
from mlx_lm import load, generate
# Load the model and tokenizer
model, tokenizer = load("nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-4bit")
# Create a prompt
prompt = "hello"
# Apply the chat template if available
if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
# Generate the response
response = generate(model, tokenizer, prompt=prompt, verbose=True)
# Print the response
print(response)
```
---
### About the Model
The **nidum/Nidum-Llama-3.2-3B-Uncensored-MLX-4bit** model was converted to MLX format from **nidum/Nidum-Llama-3.2-3B-Uncensored** using **mlx-lm version 0.19.2**, providing the following benefits:
- **Smaller Memory Footprint**: Ideal for environments with limited hardware resources.
- **High Performance**: Retains the advanced capabilities of the original model while optimizing inference speed and efficiency.
- **Plug-and-Play Compatibility**: Easily integrate with the **mlx-lm** ecosystem for seamless deployment.
---
### Use Cases
- **Technical Problem Solving**
- **Research and Educational Assistance**
- **Open-Ended Q&A**
- **Creative Writing and Ideation**
- **Long-Context Dialogues**
- **Unrestricted Knowledge Exploration**
---
### Datasets and Fine-Tuning
The model inherits the fine-tuned capabilities of its predecessor, **Nidum-Llama-3.2-3B-Uncensored**, including:
- **Uncensored Data**: Ensures detailed and uninhibited responses.
- **RAG-Based Fine-Tuning**: Optimizes retrieval-augmented generation for information-intensive tasks.
- **Math-Instruct Data**: Tailored for precise mathematical reasoning.
- **Long-Context Fine-Tuning**: Enhanced coherence and relevance in extended interactions.
---
### Quantized Model Download
The **MLX-4bit** version is highly efficient, maintaining a balance between precision and memory usage.
---
#### Benchmark
| **Benchmark** | **Metric** | **LLaMA 3B** | **Nidum 3B** | **Observation** |
|-------------------|-----------------------------------|--------------|--------------|-----------------------------------------------------------------------------------------------------|
| **GPQA** | Exact Match (Flexible) | 0.3 | 0.5 | Nidum 3B demonstrates significant improvement, particularly in **generative tasks**. |
| | Accuracy | 0.4 | 0.5 | Consistent improvement, especially in **zero-shot** scenarios. |
| **HellaSwag** | Accuracy | 0.3 | 0.4 | Better performance in **common sense reasoning** tasks. |
| | Normalized Accuracy | 0.3 | 0.4 | Enhanced ability to understand and predict context in sentence completion. |
| | Normalized Accuracy (Stderr) | 0.15275 | 0.1633 | Slightly improved consistency in normalized accuracy. |
| | Accuracy (Stderr) | 0.15275 | 0.1633 | Shows robustness in reasoning accuracy compared to LLaMA 3B. |
---
### Insights:
1. **Compact Efficiency**: The MLX-4bit model ensures high performance with reduced resource usage.
2. **Enhanced Usability**: Optimized for seamless integration with lightweight deployment scenarios.
---
### Contributing
We invite contributions to further enhance the **MLX-4bit** model's capabilities. Reach out to us for collaboration opportunities.
---
### Contact
For inquiries, support, or feedback, email us at **[email protected]**.
---
### Explore the Future
Embrace the power of innovation with **Nidum-Llama-3.2-3B-Uncensored-MLX-4bit**—the ideal blend of performance and efficiency.
--- |