update readme

README.md

@@ -6,7 +6,7 @@ datasets:
 - tiiuae/falcon-refinedweb
 ---
 
-# Model Card for Sindibad-7B
+# Model Card for Falcon-Mamba-7B
 
 
 
@@ -29,7 +29,7 @@ datasets:
 - **Model type:** Causal decoder-only
 - **Architecture:** Mamba
 - **Language(s) (NLP):** Mainly English
-- **License:** TII Sindibad License 2.0
+- **License:** TII Falcon-Mamba License 2.0
 
 ### Model Source
 
@@ -49,8 +49,8 @@ Find below some example scripts on how to use the model in `transformers` (Make
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM
 
-tokenizer = AutoTokenizer.from_pretrained("tiiuae/sindibad-7b")
-model = AutoModelForCausalLM.from_pretrained("tiiuae/sindibad-7b")
+tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-mamba-7b")
+model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-mamba-7b")
 
 input_text = "Question: How many hours in one day? Answer: "
 input_ids = tokenizer(input_text, return_tensors="pt").input_ids
@@ -70,8 +70,8 @@ print(tokenizer.decode(outputs[0]))
 # pip install accelerate
 from transformers import AutoTokenizer, AutoModelForCausalLM
 
-tokenizer = AutoTokenizer.from_pretrained("tiiuae/sindibad-7b")
-model = AutoModelForCausalLM.from_pretrained("tiiuae/sindibad-7b", device_map="auto")
+tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-mamba-7b")
+model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-mamba-7b", device_map="auto")
 
 input_text = "Question: How many hours in one day? Answer: "
 input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
@@ -94,8 +94,8 @@ print(tokenizer.decode(outputs[0]))
 import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
 
-tokenizer = AutoTokenizer.from_pretrained("tiiuae/sindibad-7b")
-model = AutoModelForCausalLM.from_pretrained("tiiuae/sindibad-7b", device_map="auto", torch_dtype=torch.float16)
+tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-mamba-7b")
+model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-mamba-7b", device_map="auto", torch_dtype=torch.float16)
 
 input_text = "Question: How many hours in one day? Answer: "
 input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
@@ -115,8 +115,8 @@ print(tokenizer.decode(outputs[0]))
 # pip install bitsandbytes accelerate
 from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
 
-tokenizer = AutoTokenizer.from_pretrained("tiiuae/sindibad-7b")
-model = AutoModelForCausalLM.from_pretrained("tiiuae/sindibad-7b", device_map="auto", quantization_config=BitsAndBytesConfig(load_in_4bit=True))
+tokenizer = AutoTokenizer.from_pretrained("tiiuae/falcon-mamba-7b")
+model = AutoModelForCausalLM.from_pretrained("tiiuae/falcon-mamba-7b", device_map="auto", quantization_config=BitsAndBytesConfig(load_in_4bit=True))
 
 input_text = "Question: How many hours in one day? Answer: "
 input_ids = tokenizer(input_text, return_tensors="pt").input_ids.to("cuda")
@@ -143,7 +143,7 @@ Overall, the data sources included RefinedWeb-English, Refined-Multilingual (lat
 The data was tokenized with the Falcon-[7B](https://huggingface.co/tiiuae/falcon-7B)/[11B](https://huggingface.co/tiiuae/falcon-11B) tokenizer.
 
 ## Training Procedure
-Sindibad-7B was trained on 256 H100 80GB GPUs for the majority of the training, using a 3D parallelism strategy (TP=1, PP=1, DP=256) combined with ZeRO.
+Falcon-Mamba-7B was trained on 256 H100 80GB GPUs for the majority of the training, using a 3D parallelism strategy (TP=1, PP=1, DP=256) combined with ZeRO.
 
 #### Training Hyperparameters
 
@@ -193,7 +193,7 @@ Refer to our technical report for more details about performance evaluation.
 
 ## Model Architecture and Objective
 
-Sindibad-7B is a causal decoder-only model trained on a causal language modeling task (i.e., predict the next token).
+Falcon-Mamba-7B is a causal decoder-only model trained on a causal language modeling task (i.e., predict the next token).
 
 The model is based on the Mamba architecture ([Gu et al., 2023](https://arxiv.org/abs/2312.00752)).
 
@@ -209,11 +209,11 @@ The model is based on the Mamba architecture ([Gu et al., 2023](https://arxiv.or
 
 ### Hardware
 
-Sindibad-7B was trained on AWS SageMaker, using on average 256 H100 80GB GPUs in 32 p5 instances. 
+Falcon-Mamba-7B was trained on AWS SageMaker, using on average 256 H100 80GB GPUs in 32 p5 instances. 
 
 ### Software
 
-Sindibad-7B was trained an internal distributed training codebase, Gigatron. It uses a 3D parallelism approach combined with ZeRO, high-performance Triton kernels.
+Falcon-Mamba-7B was trained an internal distributed training codebase, Gigatron. It uses a 3D parallelism approach combined with ZeRO, high-performance Triton kernels.
 
 # Citation