File size: 3,916 Bytes
737ed9a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 |
---
base_model: tiiuae/Falcon3-10B-Instruct
library_name: transformers
license: other
license_name: falcon-llm-license
license_link: https://falconllm.tii.ae/falcon-terms-and-conditions.html
tags:
- bitnet
- falcon3
---
# Table of Contents
0. [TL;DR](#TL;DR)
1. [Model Details](#model-details)
2. [Training Details](#training-details)
3. [Usage](#usage)
4. [Evaluation](#evaluation)
5. [Citation](#citation)
# TL;DR
# Model Details
## Model Description
- **Developed by:** [https://www.tii.ae](https://www.tii.ae)
- **Model type:** Causal decoder-only - instruct / chat version
- **Architecture:** Pure-transformer - 1.58bit version
- **Language(s) (NLP):** Mainly English
- **License:** TII Falcon License 2.0
# Training details
The model has been trained following the training strategies from the recent [1-bit LLM HF blogpost](https://huggingface.co/blog/1_58_llm_extreme_quantization) and [1-bit LLM paper](https://github.com/microsoft/unilm/blob/master/bitnet/The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ.pdf).
For more details about the training protocol of this model, please refer to the Falcon-3 technical report, section *Compression*.
# Usage
Currently to use this model you can rely on [BitNet](https://github.com/microsoft/BitNet) library. You can also play with the model using the [falcon-1.58bit playground](https://huggingface.co/spaces/tiiuae/falcon3-1.58bit-playground) (only for the 7B instruct version).
## BitNet
```
git clone https://github.com/microsoft/BitNet && cd BitNet
pip install -r requirements.txt
huggingface-cli download tiiuae/Falcon3-10B-Instruct-1.58bit-GGUF ggml-model-i2_s.gguf --local-dir models/Falcon3-10B-1.58bit/
python run_inference.py -m models/Falcon3-10B-1.58bit/ggml-model-i2_s.gguf -p "You are a helpful assistant" -cnv
```
# Evaluation
We report in the following table our internal pipeline benchmarks:
**Note evaluation results are normalized score from v2 leaderboard tasks - reported results of original models in the blogpost are raw scores**
<table border="1" style="width: 100%; text-align: center; border-collapse: collapse;">
<colgroup>
<col style="width: 10%;">
<col style="width: 10%;">
<col style="background-color: rgba(80, 15, 213, 0.5); width: 7%;">
</colgroup>
<thead>
<tr>
<th>Benchmark</th>
<th>Llama3-8B-1.58-100B-tokens</th>
<th>Falcon3-10B-Instruct-1.58bit</th>
</tr>
</thead>
<tbody>
<tr>
<td>IFEval</td>
<td>17.91</td>
<td><b>54.37</b></td>
</tr>
<tr>
<td>MUSR</td>
<td><b>4.87</b></td>
<td>2.57</td>
</tr>
<tr>
<td>GPQA</td>
<td>1.83</td>
<td><b>4.27</b></td>
</tr>
<tr>
<td>BBH</td>
<td>5.36</td>
<td><b>6.59</b></td>
</tr>
<tr>
<td>MMLU-PRO</td>
<td>2.78</td>
<td><b>6.62</b></td>
</tr>
<tr>
<td>MATH</td>
<td>0.26</td>
<td><b>2.44</b></td>
</tr>
<tr>
<td>Average</td>
<td>5.5</td>
<td><b>12.81</b></td>
</tr>
</tbody>
</table>
## Useful links
- View our [release blogpost](https://huggingface.co/blog/falcon3).
- Feel free to join [our discord server](https://discord.gg/fwXpMyGc) if you have any questions or to interact with our researchers and developers.
## Citation
If the Falcon3 family of models were helpful to your work, feel free to give us a cite.
```
@misc{Falcon3,
title = {The Falcon 3 Family of Open Models},
author = {Falcon-LLM Team},
month = {December},
year = {2024}
}
``` |