small fixes

README.md

@@ -56,7 +56,7 @@ Training a multilingual 176 billion parameters model in the open
 
 [BigScience](https://bigscience.huggingface.co) is a open and collaborative workshop around the study and creation of very large language models gathering more than 1000 researchers around the worlds. You can find more information on the main website at https://bigscience.huggingface.co.
 
-The training of BigScience’s main model started on **March 11, 2022 11:42am PST** and will last 3-4 months on the 416 A100 GPUs of the Jean Zay public supercomputer
+The training of BigScience’s main model started on **March 11, 2022 11:42am PST** and will continue for 3-4 months on 384 A100 80GB GPUs of the Jean Zay public supercomputer
 
 You can follow the training at [https://twitter.com/BigScienceLLM](https://twitter.com/BigScienceLLM)
 
@@ -75,16 +75,16 @@ You can follow the training at [https://twitter.com/BigScienceLLM](https://twitt
  
 - Multilingual: 46 languages: Full list is here: [https://bigscience.huggingface.co/blog/building-a-tb-scale-multilingual-dataset-for-language-modeling](https://bigscience.huggingface.co/blog/building-a-tb-scale-multilingual-dataset-for-language-modeling)
 - 341.6 billion tokens (1.5 TB of text data)
-- Tokenizer vocabulary: 250 680 tokens
+- Tokenizer vocabulary: 250,680 tokens
 - More information:
     - Blog post detailing the design choices during the dataset creation: [https://bigscience.huggingface.co/blog/building-a-tb-scale-multilingual-dataset-for-language-modeling](https://bigscience.huggingface.co/blog/building-a-tb-scale-multilingual-dataset-for-language-modeling)
 
 ### **The engineering side**
 
-- number of GPU used for the training: 384 A100 GPU with 80 Gb of memory each
+- number of GPU used for the training: 384 A100 GPU with 80 GB of memory each
 - one copy of the model takes 48 GPUs (using 60 GB of memory on each GPU)
-- checkpoint size: only the bf16 weights are 329GB, the full checkpoint with optimizer states is 2.3TB
-- training throughput: about 150 TFLOPs
+- checkpoint size: the bf16 weights are 329GB, the full checkpoint with optimizer states is 2.3TB
+- training throughput: ~150 TFLOPs
 - estimated training time: 3-4 months depending on throughput and unexpected events
 - **More information**:
     - Blog post on the hardware/engineering side: [https://bigscience.huggingface.co/blog/which-hardware-to-train-a-176b-parameters-model](https://bigscience.huggingface.co/blog/which-hardware-to-train-a-176b-parameters-model)