Update README.md

README.md

@@ -2,6 +2,7 @@
 license: mit
 language:
 - en
+- ar
 base_model:
 - qwen2-VL-7B
 pipeline_tag: visual-question-answering
@@ -10,7 +11,11 @@ tags:
 - Arabic
 ---
 
-# AIN: The Arabic INclusive Large Multimodal Model
+
+<div style="display: flex; align-items: center;">
+  <img src="assets_hf/AIN.png" width="5%" alt="logo" style="margin-right: 10px;" />
+  <h1 style="margin: 0; font-size: 32px;";">AIN: The Arabic INclusive Large Multimodal Model</h1>
+</div>
 
 [Ahmed Heakl](https://huggingface.co/ahmedheakl) <sup> * </sup> &nbsp;
 [Sara Ghaboura](https://huggingface.co/SLMLAH) <sup> * </sup> &nbsp;
@@ -23,45 +28,222 @@ tags:
 <em> <sup> *Equal Contribution  </sup> </em>
 <br>
 #### **Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), UAE**
-[![arXiv](https://img.shields.io/badge/arXiv-2502.0094-3399FF)](https://arxiv.org/abs/2502.00094)&nbsp;
-[![Our Page](https://img.shields.io/badge/Visit-Our%20Page-8C7AFF?style=flat)](https://mbzuai-oryx.github.io/AIN/)&nbsp;
-[![GitHub issues](https://img.shields.io/github/issues/mbzuai-oryx/Camel-Bench?color=FFF359&label=issues&style=flat)](https://github.com/mbzuai-oryx/AIN/issues)&nbsp;
-[![GitHub stars](https://img.shields.io/github/stars/mbzuai-oryx/AIN?color=FF6A07&style=flat)](https://github.com/mbzuai-oryx/AIN/stargazers)&nbsp;
+[![arXiv](https://img.shields.io/badge/arXiv-2502.0094-3399FF)](https://arxiv.org/abs/2502.00094)
+[![Our Page](https://img.shields.io/badge/Visit-Our%20Page-8C7AFF?style=flat)](https://mbzuai-oryx.github.io/AIN/)
+[![GitHub issues](https://img.shields.io/github/issues/mbzuai-oryx/Camel-Bench?color=FFF359&label=issues&style=flat)](https://github.com/mbzuai-oryx/AIN/issues)
+[![GitHub stars](https://img.shields.io/github/stars/mbzuai-oryx/AIN?color=FF6A07&style=flat)](https://github.com/mbzuai-oryx/AIN/stargazers)
 [![GitHub license](https://img.shields.io/github/license/mbzuai-oryx/Camel-Bench?color=FF6666)](https://github.com/mbzuai-oryx/AIN/blob/main/LICENSE)
 
-<div align="center">
+---
 
-  <br>
-  <br>
-   &emsp; &emsp;  &emsp;
-  <img src="https://github.com/user-attachments/assets/29421075-ec74-4843-ad8a-8bd9dfd535d6" alt="chatbot" width="30px" />
-  &emsp; <a href="https://huggingface.co/spaces/ahmedheakl/AIN-Arabic-VLM" target="_blank">AIN Chatbot</a>
-  <img src="https://github.com/user-attachments/assets/451fd639-7cb7-4e77-b7ed-c3678bf980e3" alt="telegram" width="25px" />
-  &emsp; <a href="https://t.me/arabicvlm_bot" target="_blank">AIN Telegram</a>
-  <img src="https://github.com/user-attachments/assets/35bd262d-55d1-45e9-8382-e44567b09102" alt="WhatsApp" width="25px" />
-  &emsp; <a href="https://wa.me/46738645096" target="_blank">AIN WhatsApp</a> 
-  <br>
-  <br>
-</div>
 
+<div class="abstract-container">
+  <h2>Abstract</h2>
+     <div class="abstract-content">
+        <p>
+           Amid the swift progress of large language models (LLMs) and their evolution into large multimodal models (LMMs), significant strides have been made in high-resource languages such as English and Chinese. While Arabic LLMs have seen notable progress, Arabic LMMs remain largely unexplored, often narrowly focusing on a few specific aspects of the language and visual understanding. To bridge this gap, we introduce <b><em>AIN - the Arabic Inclusive Multimodal Model-</em></b> designed to excel across diverse domains.
+           AIN is an English-Arabic <b>bilingual LMM</b> designed to excel in English and Arabic, leveraging carefully constructed <b>3.6 million</b> high-quality Arabic-English multimodal data samples. AIN demonstrates state-of-the-art Arabic performance, while also possessing strong English-language visual capabilities.
+        </p>
+    </div>
+ </div>
 
 
+  
+ ## 🌟 Key Features
+ - The **first Arabic-centric inclusive Large Multimodal Model (LMM)** trained on **3.6M samples**.
+ - Includes **35% authentic Arabic data** within its Arabic data subset.
+ - Achieves **superior performance compared to open- and closed-source models** (e.g., GPT-4o) and open-source models (e.g., Qwen2-VL-7B) across tasks such as OCR and specialized domains.
+ - Demonstrates **robust bilingual capabilities** (Arabic/English), **validated** through **comprehensive testing** and **human evaluation** across 17 Arab countries.
+ - Exhibits **advanced cultural understanding** and domain expertise in fields such as **medical imaging**, **agriculture**, and **scientific visualization**.
+
+
+<p align="center">
+   <img src="assets_hf/intro_bar.png" width="50%" alt="intro_bar"  style="margin-right: 2px";/>
+   <h6>
+       <em>  <b>Figure 1.</b> Comparative performance of AIN-7B against other models across key domains, including OCR & Document Understanding, Remote Sensing, Agricultural Understanding, and overall performance across all domains. </em>
+   </h6>
+</p> 
+
+<p align="center" >
+   <img src="assets_hf/radar_chart.png" width="35%" alt="radar_chart"  style="margin-right: 2px";/>
+ <h6>
+       <em>  <b>Figure 2.</b> showcases a comprehensive performance analysis of AIN-7B across CAMEL-Bench domains, comparing it with prominent closed-source models as well as open-source counterparts. <strong>OCR:</strong> "OCR & Document Understanding",  <strong>Video:</strong> "General Video & Multi-Image Understanding",  <strong>RS:</strong> "Remote Sensing Understanding", <strong>CDT:</strong> "Chart, Diagram & Table Understanding",  <strong>Agro.:</strong> "Agricultural Image Understanding", <strong>Cultural:</strong> "Cultural-Specific Understanding", <strong>Medical:</strong> "Medical Image Understanding".
+       </em> 
+ </h6>
+  
 ---
+## ⚖️ Quantitative Evaluation and Results
+AIN demonstrates state-of-the-art performance across diverse domains, surpassing both open- and closed-source models. Notably, it achieves an aggregate performance score of 63.77%, with significant gains in OCR, remote sensing, and agricultural image understanding.
 
+<div align="center" >
+<table>
+    <caption>
+        <h6>
+        <strong>Table 1. Performance comparison of AIN and different closed- and open-source LMMs across CAMEL-Bench domains.</strong> 
+        <br> <em>Best performance is marked with 🥇; second-best is 🥈.</em>
+            <strong>OCR</strong>: "OCR & Document Understanding", 
+            <strong>Video</strong>: "General Video & Multi-Image Understanding", 
+            <strong>RS</strong>: "Remote Sensing Understanding", 
+            <strong>CDT</strong>: "Chart, Diagram & Table Understanding", 
+            <strong>Agro.</strong>: "Agricultural Image Understanding", 
+            <strong>Cult.</strong>: "Cultural-Specific Understanding",  
+            <strong>Med.</strong>: "Medical Image Understanding".
+        </h6>
+    </caption>
+    <thead>
+        <tr style="background-color: #e0e0e0;">
+            <th>Models</th>
+            <th>VQA</th>
+            <th>OCR</th>
+            <th>Video</th>
+            <th>RS</th>
+            <th>CDT</th>
+            <th>Agro.</th>
+            <th>Cult.</th>
+            <th>Med.</th>
+            <th style="background-color: #d0d0d0;">Total</th>
+        </tr>
+    </thead>
+    <tbody>
+        <tr>
+            <td>GPT-4o</td>
+            <td>🥈55.15</td>
+            <td>🥈54.98</td>
+            <td>🥇69.65</td>
+            <td>🥈27.36</td>
+            <td>🥈62.35</td>
+            <td>🥈80.75</td>
+            <td>🥇80.86</td>
+            <td>🥇49.91</td>
+            <td style="background-color: #d0d0d0;">🥈60.13</td>
+        </tr>
+        <tr>
+            <td>GPT-4o-mini</td>
+            <td>48.83</td>
+            <td>39.38</td>
+           <td>🥈66.28</td>
+            <td>16.93</td>
+            <td>56.37</td>
+            <td>78.80</td>
+            <td>65.92</td>
+           <td>🥈47.37</td>
+            <td style="background-color: #d0d0d0;">52.49</td>
+        </tr>
+        <tr>
+            <td>Gemini-1.5-Pro</td>
+            <td>46.68</td>
+            <td>28.68</td>
+            <td>42.95</td>
+            <td>17.07</td>
+            <td>47.06</td>
+            <td>72.14</td>
+            <td>56.24</td>
+            <td>33.78</td>
+            <td style="background-color: #d0d0d0;">52.38</td>
+        </tr>
+        <tr>
+            <td>Gemini-1.5-flash</td>
+            <td>45.59</td>
+            <td>27.58</td>
+            <td>53.31</td>
+            <td>14.95</td>
+            <td>48.26</td>
+            <td>76.07</td>
+            <td>46.54</td>
+            <td>42.87</td>
+            <td style="background-color: #d0d0d0;">44.40</td>
+        </tr>
+        <tr>
+            <td>InternVL-8B </td>
+            <td>30.41 </td>
+            <td>15.91 </td>
+            <td>51.42 </td>
+            <td>5.36 </td>
+            <td>30.27 </td>
+            <td>44.47 </td>
+            <td>20.88 </td>
+            <td>29.48 </td>
+            <td style="background-color: #d0d0d0;">28.52 </td>
+        </tr>
+        <tr>
+            <td>InternVL2.5-1B </td>
+            <td>27.22 </td>
+            <td>19.45 </td>
+            <td>38.20 </td>
+            <td>3.39 </td>
+            <td>30.75 </td>
+            <td>39.53 </td>
+            <td>35.68 </td>
+            <td>21.27 </td>
+            <td style="background-color: #d0d0d0;">26.94 </td>
+        </tr>
+        <tr>
+            <td>Qwen-VL-2B </td>
+            <td>41.02 </td>
+            <td>22.93 </td>
+            <td>38.90 </td>
+            <td>12.56 </td>
+            <td>27.83 </td>
+            <td>52.02 </td>
+            <td>34.28 </td>
+            <td>29.12 </td>
+            <td style="background-color: #d0d0d0;">32.33 </td>
+        </tr>
+        <tr>
+            <td>AIN-7B <em>(ours)</em> </td>
+           <td>🥇56.78 </td>
+            <td>🥇72.35 </td>
+            <td>64.09 </td>
+            <td>🥇45.92 </td>
+           <td>🥇64.10 </td>
+            <td>🥇85.05 </td>
+           <td>🥈78.09 </td>
+            <td>43.77 </td>
+            <td style="background-color: #d0d0d0;">🏆63.77 </td>
+        </tr>
+    </tbody>
+</table>
+   </div>         
+   
+---
+## 🎯 Qualitative Evaluation
+The qualitative evaluation showcases AIN's advanced capabilities in handling diverse, complex tasks, including OCR, medical imaging, remote sensing, and cultural-specific understanding, with remarkable precision and contextual relevance. Unlike GPT-4o and LLaVA, AIN demonstrates superior performance in identifying intricate details and maintaining accuracy across varied query formats and multi-domain challenges.
+
+<div align="center">
+  <img src="assets_hf/qualitative.png" width="50%" alt="qualitative" />
+     <h6>
+       <em>  <b>Figure 3.</b> Qualitative examples showcasing AIN-7B’s capabilities across various domains, including general VQA, OCR & Document Understanding, Remote Sensing, Medical Imaging, Agricultural Understanding, and Cultural-Specific tasks. </em>
+    </h6>
+</div>
+
+---
+## 🧐 Data Verification and Toxicity Filtering
+A multi-step verification pipeline was implemented to ensure high-quality translations and safe visual data. Translation accuracy was assessed through human evaluation, where native Arabic speakers rated outputs against reference translations, and semantic similarity checks were conducted using **LaBSE**. Additionally, translated samples were reverse-translated and validated using **BLEU, METEOR, and ROUGE scores** to measure correctness, correlation, and overlap. For visual data, toxicity filtering was applied using **LLavaGuard’s safety policies and GPT-4o**, identifying and removing unsafe content related to violence, substance abuse, and harmful imagery, ensuring compliance with ethical AI standards.
+
+<p align="center">
+   <img src="assets_hf/verify_pipeline.png" width="45%" alt="verify"  style="margin-right: 2px";/>
+    <h6>
+       <em>  <b>Figure 4.</b> Data verification and filtering pipeline for textual and visual data, ensuring high-quality training data through semantic similarity checks, translation quality evaluations, and toxicity screening for safety compliance. </em>
+    </h6>
+</p> 
+<p align="center">
+   <img src="assets_hf/toxicity.png" width=30%" alt="verify"  style="margin-right: 2px";/>
+    <h6>
+       <em>  <b>Figure 5.</b> Distribution of visual data toxicity filtering results, showing that 95% of the data is classified as safe, while 5% is identified as unsafe due to categories like weapons or substance abuse, violence, and animal cruelty. </em>
+   </h6>
+</p> 
 
 ---
 
 ##  License
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
-<br>
-<br>
+
 
 ## 💬 Contact us
 For questions or suggestions, feel free to reach out to us on [GitHub Discussions](https://github.com/mbzuai-oryx/AIN/discussions).
 
 ---
 
-
 If you use AIN in your research, please cite our work as follows: 
 
 ```