Update README.md
Browse files
README.md
CHANGED
@@ -11,7 +11,7 @@ base_model:
|
|
11 |
<img src="https://i.ibb.co/CmPSSpq/Screenshot-2024-10-06-at-9-45-06-PM.png" alt="Arcee Meraj Mini" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
|
12 |
</div>
|
13 |
|
14 |
-
Following the release of [Arcee Meraj](https://meraj.arcee.ai/), our enterprise globally top-performing Arabic LLM, we are thrilled to unveil Arcee Meraj Mini. This open-source model, meticulously fine-tuned from [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), is expertly designed for both Arabic and English. This model has undergone rigorous evaluation across multiple benchmarks in both languages, demonstrating top-tier performance in Arabic and competitive results in English. Arcee Meraj Mini’s primary objective is to enhance Arabic capabilities while maintaining robust English language proficiency. Benchmark results confirm that Arcee Meraj Mini excels in Arabic, with English performance comparable to leading models — perfectly aligning with our vision for balanced bilingual strength.
|
15 |
|
16 |
## Technical Details
|
17 |
Below is an overview of the key stages in Meraj Mini’s development:
|
@@ -20,7 +20,7 @@ Below is an overview of the key stages in Meraj Mini’s development:
|
|
20 |
2. **Initial Training:** We train the Qwen2.5 model with 7 billion parameters using these high-quality datasets in both languages. This allows the model to handle diverse linguistic patterns from over 500 million tokens, ensuring strong performance in Arabic and English tasks.
|
21 |
3. **Iterative Training and Post-Training:** Iterative training and post-training iterations refine the model, enhancing its accuracy and adaptability to ensure it can perform well across varied tasks and language contexts.
|
22 |
4. **Evaluation:** Arcee Meraj Mini is based on training and evaluating 15 different variants to explore optimal configurations, with assessments done on both Arabic and English benchmarks and leaderboards. This step ensures the model is robust in handling both general and domain-specific tasks.
|
23 |
-
5. **Final Model Creation:** We select the best-performing variant and use the [MergeKit](https://arxiv.org/pdf/2403.13257) library to merge the configurations, resulting in the final Arcee Meraj Mini model. This model is not only optimized for language understanding
|
24 |
|
25 |
With this process, Arcee Meraj Mini is crafted to be more than just a general-purpose language model—it’s an adaptable tool, ready to be fine-tuned for specific industries and applications, empowering users to extend its capabilities for domain-specific tasks.
|
26 |
## Capabilities and Use Cases
|
@@ -44,14 +44,14 @@ Arcee Meraj Mini is capable of solving a wide range of language tasks, including
|
|
44 |
#### Open Arabic LLM Leaderboard (OALL) Benchmarks
|
45 |
Arcee Meraj Mini model consistently outperforms state-of-the-art models on most of the Open Arabic LLM Leaderboard (OALL) benchmarks, highlighting its improvements and effectiveness in Arabic language content, and securing the top performing position on average among the other models.
|
46 |
<div align="center">
|
47 |
-
<img src="https://i.ibb.co/2FSdtY8/Table-1.png" alt="Arcee Meraj Mini Open Arabic LLM
|
48 |
</div>
|
49 |
<div align="center">
|
50 |
-
<img src="https://i.ibb.co/2FGrYfb/Table-2.png" alt="Arcee Meraj Mini Open Arabic LLM
|
51 |
</div>
|
52 |
|
53 |
#### Translated MMLU
|
54 |
-
We focused on the multilingual MMLU dataset, as distributed through the LM Evaluation Harness repository, to compare the multilingual strength of different models for this benchmark. Arcee Meraj Mini along with the base model, Qwen2.5-7B-Instruct
|
55 |
<div align="center">
|
56 |
<img src="https://i.ibb.co/RbvYYQ1/translated-mmlu.png" alt="Arcee Meraj Mini Trnalsated MMLU" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
|
57 |
</div>
|
@@ -59,10 +59,9 @@ We focused on the multilingual MMLU dataset, as distributed through the LM Evalu
|
|
59 |
|
60 |
## Acknowledgements
|
61 |
|
62 |
-
We are grateful to the open
|
63 |
|
64 |
## Future Directions
|
65 |
|
66 |
-
As we
|
67 |
|
68 |
-
If you're interested, join the Arcee Globe community discord channel to collaborate: https://discord.gg/G3aEAQuP
|
|
|
11 |
<img src="https://i.ibb.co/CmPSSpq/Screenshot-2024-10-06-at-9-45-06-PM.png" alt="Arcee Meraj Mini" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
|
12 |
</div>
|
13 |
|
14 |
+
Following the release of [Arcee Meraj](https://meraj.arcee.ai/), our enterprise's globally top-performing Arabic LLM, we are thrilled to unveil Arcee Meraj Mini. This open-source model, meticulously fine-tuned from [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct), is expertly designed for both Arabic and English. This model has undergone rigorous evaluation across multiple benchmarks in both languages, demonstrating top-tier performance in Arabic and competitive results in English. Arcee Meraj Mini’s primary objective is to enhance Arabic capabilities while maintaining robust English language proficiency. Benchmark results confirm that Arcee Meraj Mini excels in Arabic, with English performance comparable to leading models — perfectly aligning with our vision for balanced bilingual strength.
|
15 |
|
16 |
## Technical Details
|
17 |
Below is an overview of the key stages in Meraj Mini’s development:
|
|
|
20 |
2. **Initial Training:** We train the Qwen2.5 model with 7 billion parameters using these high-quality datasets in both languages. This allows the model to handle diverse linguistic patterns from over 500 million tokens, ensuring strong performance in Arabic and English tasks.
|
21 |
3. **Iterative Training and Post-Training:** Iterative training and post-training iterations refine the model, enhancing its accuracy and adaptability to ensure it can perform well across varied tasks and language contexts.
|
22 |
4. **Evaluation:** Arcee Meraj Mini is based on training and evaluating 15 different variants to explore optimal configurations, with assessments done on both Arabic and English benchmarks and leaderboards. This step ensures the model is robust in handling both general and domain-specific tasks.
|
23 |
+
5. **Final Model Creation:** We select the best-performing variant and use the [MergeKit](https://arxiv.org/pdf/2403.13257) library to merge the configurations, resulting in the final Arcee Meraj Mini model. This model is not only optimized for language understanding but also serves as a starting point for domain adaptation in different areas.
|
24 |
|
25 |
With this process, Arcee Meraj Mini is crafted to be more than just a general-purpose language model—it’s an adaptable tool, ready to be fine-tuned for specific industries and applications, empowering users to extend its capabilities for domain-specific tasks.
|
26 |
## Capabilities and Use Cases
|
|
|
44 |
#### Open Arabic LLM Leaderboard (OALL) Benchmarks
|
45 |
Arcee Meraj Mini model consistently outperforms state-of-the-art models on most of the Open Arabic LLM Leaderboard (OALL) benchmarks, highlighting its improvements and effectiveness in Arabic language content, and securing the top performing position on average among the other models.
|
46 |
<div align="center">
|
47 |
+
<img src="https://i.ibb.co/2FSdtY8/Table-1.png" alt="Arcee Meraj Mini Open Arabic LLM Leaderboard (OALL) - table 1" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
|
48 |
</div>
|
49 |
<div align="center">
|
50 |
+
<img src="https://i.ibb.co/2FGrYfb/Table-2.png" alt="Arcee Meraj Mini Open Arabic LLM Leaderboard (OALL) - table 2" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
|
51 |
</div>
|
52 |
|
53 |
#### Translated MMLU
|
54 |
+
We focused on the multilingual MMLU dataset, as distributed through the LM Evaluation Harness repository, to compare the multilingual strength of different models for this benchmark. Arcee Meraj Mini along with the base model, Qwen2.5-7B-Instruct outperforms the other models, showcasing these models’ superior performance compared to the other state-of-the-art models.
|
55 |
<div align="center">
|
56 |
<img src="https://i.ibb.co/RbvYYQ1/translated-mmlu.png" alt="Arcee Meraj Mini Trnalsated MMLU" style="border-radius: 10px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2), 0 6px 20px 0 rgba(0, 0, 0, 0.19); max-width: 100%; height: auto;">
|
57 |
</div>
|
|
|
59 |
|
60 |
## Acknowledgements
|
61 |
|
62 |
+
We are grateful to the open-source AI community for their continuous contributions and to the Qwen team for their foundational efforts on the Qwen2.5 model series.
|
63 |
|
64 |
## Future Directions
|
65 |
|
66 |
+
As we release the Arcee Meraj Mini to the public, we invite researchers, developers, and businesses to engage with the Arcee Meraj Mini model, particularly in enhancing support for the Arabic language and fostering domain adaptation. We are committed to advancing open-source AI technology and invite the community to explore, contribute, and build upon Arcee Meraj Mini.
|
67 |
|
|