tiiuae
/

Falcon3-Mamba-7B-Instruct

Text Generation

Inference Endpoints

Model card Files Files and versions Community

JingweiZuo commited on 25 days ago

Commit

fc299b4

·

verified ·

1 Parent(s): 6299176

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -33,7 +33,7 @@ Falcon3-Mamba-7B-Instruct supports a context length up to 32K and was mainly tra
   - state_size: 16
   - 32k context length
   - 65k vocab size
-- Continue Pretrained from [Falcon Mamba 7B](https://huggingface.co/tiiuae/falcon-mamba-7b), with another 1500 Gigatokens of data comprising of web, code, STEM and high quality data.
 - Postrained on 1.2 million samples of STEM, conversations, code, and safety.
 - Developed by [Technology Innovation Institute](https://www.tii.ae)
 - License: TII Falcon-LLM License 2.0

   - state_size: 16
   - 32k context length
   - 65k vocab size
+- Continue Pretrained from [Falcon-Mamba-7b](https://arxiv.org/abs/2410.05355), with another 1500 Gigatokens of data consisting of web, code, STEM and high quality data.
 - Postrained on 1.2 million samples of STEM, conversations, code, and safety.
 - Developed by [Technology Innovation Institute](https://www.tii.ae)
 - License: TII Falcon-LLM License 2.0