Mxode
/

NanoLM-0.3B-Instruct-v1.1

Text2Text Generation

Model card Files Files and versions Community

Mxode commited on Sep 7, 2024

Commit

6dcd97c

·

verified ·

1 Parent(s): d1f3901

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -37,7 +37,7 @@ This is NanoLM-0.3B-Instruct-v1.1. The model currently supports both **Chinese a
 | :----------: | :------------------: | :---: | :----: | :-------: | :---: | :---: |
 | 25M         | 15M  |   MistralForCausalLM     | 12      | 312     | 12    |2K|
 | 70M         | 42M |  LlamaForCausalLM          | 12     | 576    | 9   |2K|
-| 0.3B         | 180M |  Qwen2ForCausalLM  | 12   | 896    | 14 |4K|
 | 1B     | 840M | Qwen2ForCausalLM | 18   | 1536   | 12   |4K|
 The tokenizer and model architecture of NanoLM-0.3B-Instruct-v1.1 are the same as [Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B), but the number of layers has been reduced from 24 to 12.

 | :----------: | :------------------: | :---: | :----: | :-------: | :---: | :---: |
 | 25M         | 15M  |   MistralForCausalLM     | 12      | 312     | 12    |2K|
 | 70M         | 42M |  LlamaForCausalLM          | 12     | 576    | 9   |2K|
+| **0.3B**        | **180M** |  **Qwen2ForCausalLM**  | **12**   | **896**    | **14** | **4K** |
 | 1B     | 840M | Qwen2ForCausalLM | 18   | 1536   | 12   |4K|
 The tokenizer and model architecture of NanoLM-0.3B-Instruct-v1.1 are the same as [Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B), but the number of layers has been reduced from 24 to 12.