ssmits
/

Falcon2-5.5B-multilingual

Text Generation

tiiuae/falcon-11B

Inference Endpoints

Model card Files Files and versions Community

ssmits commited on Jun 17

Commit

e9d1ee2

•

1 Parent(s): ab408ab

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -28,7 +28,7 @@ Even though [Falcon-11B](https://huggingface.co/tiiuae/falcon-11B) is trained on
 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/660c0a02cf274b3ab77dd6b7/QeaL9bOrPskustzFpjMUP.png)
 This is why the choice is made to prune 50% of the layers.
 Note that \~1B of continued pre-training (\~1M rows of 1k tokens) is still required to restore the perplexity of this model in the desired language.
-I'm planning on doing that for certain languages, depending on how much compute will be available.
 # sliced

 ![image/png](https://cdn-uploads.huggingface.co/production/uploads/660c0a02cf274b3ab77dd6b7/QeaL9bOrPskustzFpjMUP.png)
 This is why the choice is made to prune 50% of the layers.
 Note that \~1B of continued pre-training (\~1M rows of 1k tokens) is still required to restore the perplexity of this model in the desired language.
+I'm planning on doing that for certain languages when fineweb-edu-{specific_language} will be available, depending on how much compute will be available.
 # sliced