Update README.md

README.md

@@ -8,60 +8,71 @@ license: apache-2.0
 ## Model Resources
 
 - **Repository:** https://github.com/openevabyte/evabyte
-- **Blog:** https://hkunlp.github.io/blog/2024/evabyte
+- **Blog:** https://hkunlp.github.io/blog/2025/evabyte
 - **Paper:** Coming soon
 
 ## Model Details
 
-EvaByte is trained using the SambaNova SN30 RDU system with a batch size of 8M bytes and 32K context length. The training process consists of 3 phases: after pre-training on 1.2T bytes (yielding **EvaByte-6.5B-Phase1**), two independent annealing runs (100B and 200B bytes respectively) are conducted with learning rate linearly decayed from 1e-4 to 0. The resulting checkpoints are merged via model soup (**EvaByte-6.5B**), which then undergoes supervised fine-tuning (**EvaByte-6.5B-SFT**).
+EvaByte is trained using the performant SambaNova SN30 RDU system with a batch size of 8M bytes and 32K context length. The training process consists of 3 phases: after pre-training on 1.2T bytes (yielding **EvaByte-Phase1**), two independent annealing runs (100B and 200B bytes respectively) are conducted with learning rate linearly decayed from 1e-4 to 0. The resulting checkpoints are merged via model soup (**EvaByte**), which then undergoes supervised fine-tuning (**EvaByte-SFT**).
 
 | Stage | Model |
 |:----- |:-----|
-| Base (before annealing) | [EvaByte-6.5B-Phase1](https://huggingface.co/evabyte/EvaByte-6.5B-Phase1) <-- you are here |
-| Base | [EvaByte-6.5B](https://huggingface.co/evabyte/EvaByte-6.5B) |
-| SFT  | [EvaByte-6.5B-SFT](https://huggingface.co/evabyte/EvaByte-6.5B-SFT) |
+| Base (before annealing) | [EvaByte-Phase1](https://huggingface.co/evabyte/EvaByte-Phase1) <-- you are here |
+| Base | [EvaByte](https://huggingface.co/evabyte/EvaByte) |
+| SFT  | [EvaByte-SFT](https://huggingface.co/evabyte/EvaByte-SFT) |
 
 
 ## Usage
 
+**Note:** Make sure to set `trust_remote_code=True` when loading the model (or tokenizer), as our implementation includes custom code.
+
+The code snippet below demonstrates EvaByte-6.5B for completion:
+
 ```python
 from transformers import AutoTokenizer, AutoModelForCausalLM
 import torch
 
-tokenizer = AutoTokenizer.from_pretrained("evabyte/EvaByte-6.5B-Phase1", trust_remote_code=True)
-model = AutoModelForCausalLM.from_pretrained("evabyte/EvaByte-6.5B-Phase1", torch_dtype=torch.bfloat16, trust_remote_code=True).eval().to("cuda")
+# Load model and tokenizer
+tokenizer = AutoTokenizer.from_pretrained("evabyte/EvaByte-Phase1", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("evabyte/EvaByte-Phase1", torch_dtype=torch.bfloat16, trust_remote_code=True).eval().to("cuda")
 
 prompt = "The quick brown fox jumps "
 
-input_ids = tokenizer(prompt, return_tensors="pt").input_ids
-# alternatively, simply use the UTF-8 bytes. 
-# Note: the tokenizer offsets each byte by 64 and prepends the sentinel <bos>
-input_ids = torch.tensor([[1] + list(map(lambda x: x + 64, prompt.encode("utf-8")))])
+# Tokenize input
+# Option 1: standard HF tokenizer interface
+input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to("cuda")
 
-input_ids = input_ids.to("cuda")
+# Option 2: Direct UTF-8 byte encoding with offset
+# Note: Each byte is offset by 64 with <bos> prepended.
+input_ids = torch.tensor([[1] + [b + 64 for b in prompt.encode("utf-8")]]).to("cuda")
 
 # byte-by-byte generation (default)
 generation_output = model.generate(
     input_ids=input_ids, 
     max_new_tokens=32
 )
-# alternatively, use multibyte generation
+# alternatively, use faster multibyte generation
 generation_output = model.multi_byte_generate(
     input_ids=input_ids, 
     max_new_tokens=32
 )
 
+# Decode and print the output
 response = tokenizer.decode(
     generation_output[0][input_ids.shape[1]:], 
     skip_special_tokens=False,
     clean_up_tokenization_spaces=False
 )
 print(response)
+# Sample output:
+# over the lazy dog.\n\nThe quick
 ```
 
-We support two modes of generation:
-- `model.generate()`: When invoked, the model will generate one byte at a time. This is the default generation interface in the Huggingface `transformers` library.
-- `model.multi_byte_generate()`: the model will generate multiple bytes in a single step, with the implementation adapted from [Medusa](https://github.com/FasterDecoding/Medusa). This will be much faster than above and usually yields the same result under the setting of greedy decoding. `model.multi_byte_generate()` supports a subset of arguments in `model.generate()`:
+### ⚙️ Generation Modes
+
+EvaByte supports two generation interfaces:
+- `model.generate()`: The default generation method compatible with Huggingface `transformers` library. This approach generates one byte at a time and might be slow.
+- `model.multi_byte_generate()`: A faster alternative that generates multiple bytes per step and usually yields the same result as `model.generate()` under greedy decoding, with the implementation adapted from [Medusa](https://github.com/FasterDecoding/Medusa). `model.multi_byte_generate()` supports a subset of arguments in `model.generate()`:
     - `input_ids`: the input byte ids.
     - `temperature`: the temperature for sampling.
     - `max_length`: the maximum length of the generated sequence.
@@ -70,27 +81,25 @@ We support two modes of generation:
     - `top_p`: the top-p parameter for sampling.
     - `do_sample`: greedy decoding or sampling.
 
-NOTE:
-- `device_map="auto"` is not supported for > 2 GPUs
-- Decoding only supports batch size of 1 with `attention_mask=None` for now.
-- Only supports `torch_dtype=torch.bfloat16` for now.
+**Notes and Limitations:**
+- `device_map="auto"` is not supported for >2 GPUs.
+- Only batch size of 1 (with `attention_mask=None`) is supported for decoding.
+- `torch_dtype=torch.bfloat16` is required.
+- The multibyte generation `model.multi_byte_generate()` might return extra bytes after the end-of-sequence sentinel, due to the nature of the multibyte decoding. Manual truncation or cleaning may be needed.
 
 ## Bias, Risks, and Limitations
-As a pretrained base model, **EvaByte-6.5B-Phase1** has not been fine-tuned for chat or instruction following, so users should not expect reliable performance in conversational or instruction-based tasks. Like other base models, it does not incorporate any moderation mechanisms, making it possible to generate potentially harmful or inappropriate content.
+As a pretrained base model, **EvaByte-Phase1** has not been fine-tuned for chat or instruction following, so users should not expect reliable performance in conversational or instruction-based tasks. Like other base models, it does not incorporate any moderation mechanisms, making it possible to generate potentially harmful or inappropriate content.
 
 ## Evaluation 
 
-For detailed evaluation results, please refer to the [blog](https://hkunlp.github.io/blog/2024/evabyte).
+For detailed evaluation results, please refer to the [blog post](https://hkunlp.github.io/blog/2025/evabyte).
 
 ## Citation
-
-**BibTeX:**
-
-```
+```bibtex
 @misc{evabyte,
     title = {EvaByte: Efficient Byte-level Language Models at Scale},
-    url = {},
-    author = {Lin Zheng and Xueliang Zhao and Guangtao Wang and Chen Wu and David Dong and Angela Wang and Mingran Wang and Haige Bo and Tony Zhang and Changran Hu and Urmish Thakker and Lingpeng Kong},
+    url = {https://hkunlp.github.io/blog/2025/evabyte},
+    author = {Lin Zheng and Xueliang Zhao and Guangtao Wang and Chen Wu and David Dong and Angela Wang and Mingran Wang and Yun Du and Haige Bo and Tony Zhang and Changran Hu and Urmish Thakker and Lingpeng Kong},
     year = {2025}
 }
 ```