Update README.md

README.md

@@ -9,10 +9,7 @@ language:
 
 <img src="https://allenai.org/olmo/olmo-7b-animation.gif" alt="OLMo Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
-# Model Card for OLMo 1.7-7B-hf
-
-OLMo 1.7 7B is the latest version of the original [OLMo 7B](https://huggingface.co/allenai/OLMo-7B) model rocking a 24 point increase in MMLU, among other evaluations improvements, from an improved version of the Dolma dataset and staged training.
-**This version is for direct use with HuggingFace Transformers** from v4.40 on.
+# Model Card for OLMo 7B July 2024
 
 OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
 The OLMo models are trained on the [Dolma](https://huggingface.co/datasets/allenai/dolma) dataset.
@@ -23,30 +20,25 @@ We release all code, checkpoints, logs, and details involved in training these m
 The core models released in this batch are the following: 
 | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
 |------|--------|---------|-------------|-----------------|----------------|
-| [OLMo 1B](https://huggingface.co/allenai/OLMo-1B)   | 3 Trillion |16     | 2048        | 16              | 2048  |
-| [OLMo 7B](https://huggingface.co/allenai/OLMo-7B) | 2.5 Trillion   | 32     | 4096        | 32              |  2048  |
-| [OLMo 7B Twin 2T](https://huggingface.co/allenai/OLMo-7B-Twin-2T) | 2 Trillion   | 32     | 4096        | 32              |  2048  |
-| [OLMo 1.7-7B](https://huggingface.co/allenai/OLMo-1.7-7B) | 2.05 Trillion   | 32     | 4096        | 32              |  4096  |
-
-*Note: OLMo 1.7-7B also includes QKV clipping.*
+| [OLMo 1B July 2024](https://huggingface.co/allenai/OLMo-1B-0724-hf) | 3.05 Trillion   | 16     | 2048        | 16              |  4096  |
+| [OLMo 7B July 2024](https://huggingface.co/allenai/OLMo-7B-0724-hf) | 2.75 Trillion   | 32     | 4096        | 32              |  4096  |
 
 
 [Coming soon] We are releasing many checkpoints for these models, for every 1000 training steps.
-The naming convention is `step1000-tokens4B`.
+The naming convention is `stepXXX-tokensYYYB`.
 
 To load a specific model revision with HuggingFace, simply add the argument `revision`:
 ```bash
-olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1.7-7B-hf", revision="step1000-tokens4B")
+olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-7B-0424-hf", revision="step1000-tokens4B")
 ```
 
 All revisions/branches are listed in the file `revisions.txt`. 
 Or, you can access all the revisions for the models via the following code snippet:
 ```python
 from huggingface_hub import list_repo_refs
-out = list_repo_refs("allenai/OLMo-1.7-7B-hf")
+out = list_repo_refs("allenai/OLMo-7B-0424-hf")
 branches = [b.name for b in out.branches]
 ```
-A few revisions were lost due to an error, but the vast majority are present.
 
 ### Model Description
 
@@ -80,8 +72,8 @@ Install Transformers [from source](https://huggingface.co/docs/transformers/en/i
 Now, proceed as usual with HuggingFace:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1.7-7B-hf")
-tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-1.7-7B-hf")
+olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-7B-0424-hf")
+tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-7B-0424-hf")
 message = ["Language modeling is "]
 inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
 # optional verifying cuda
@@ -94,20 +86,14 @@ print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])
 Alternatively, with the pipeline abstraction:
 ```python
 from transformers import pipeline
-olmo_pipe = pipeline("text-generation", model="allenai/OLMo-1.7-7B-hf")
+olmo_pipe = pipeline("text-generation", model="allenai/OLMo-7B-0424-hf")
 print(olmo_pipe("Language modeling is "))
 >> 'Language modeling is a branch of natural language processing that aims to...'
 ```
 
-Or, you can make this slightly faster by quantizing the model, e.g. `AutoModelForCausalLM.from_pretrained("allenai/OLMo-1.7-7B-hf", torch_dtype=torch.float16, load_in_8bit=True)` (requires `bitsandbytes`).
+Or, you can make this slightly faster by quantizing the model, e.g. `AutoModelForCausalLM.from_pretrained("allenai/OLMo-7B-0424-hf", torch_dtype=torch.float16, load_in_8bit=True)` (requires `bitsandbytes`).
 The quantized model is more sensitive to typing / cuda, so it is recommended to pass the inputs as `inputs.input_ids.to('cuda')` to avoid potential issues.
 
-Note, you may see the following error if `ai2-olmo` is not installed correctly, which is caused by internal Python check naming. We'll update the code soon to make this error clearer.
-```bash
-    raise ImportError(
-ImportError: This modeling file requires the following packages that were not found in your environment: hf_olmo. Run `pip install hf_olmo`
-```
-
 ### Fine-tuning
 Model fine-tuning can be done from the final checkpoint (the `main` revision of this model) or many intermediate checkpoints. Two recipes for tuning are available.
 1. Fine-tune with the OLMo repository: