Update README.md
Browse files
README.md
CHANGED
|
@@ -2,11 +2,12 @@
|
|
| 2 |
library_name: transformers
|
| 3 |
tags: []
|
| 4 |
---
|
| 5 |
-
Input, output embedding weights are possibly not tied -
|
| 6 |
-
|
| 7 |
Some weights of LlamaForCausalLM were not initialized from the model checkpoint at ./llama3-pa/llama3-32kpa-emb-init-weight-tied and are newly initialized: ['lm_head.weight']
|
| 8 |
|
| 9 |
-
used -
|
|
|
|
| 10 |
source_model.lm_head.weight.data = source_model.model.embed_tokens.weight.data
|
| 11 |
source_model.lm_head.weight = source_model.model.embed_tokens.weight
|
| 12 |
-
source_model.tie_weights()
|
|
|
|
|
|
| 2 |
library_name: transformers
|
| 3 |
tags: []
|
| 4 |
---
|
| 5 |
+
### Input, output embedding weights are possibly not tied -
|
|
|
|
| 6 |
Some weights of LlamaForCausalLM were not initialized from the model checkpoint at ./llama3-pa/llama3-32kpa-emb-init-weight-tied and are newly initialized: ['lm_head.weight']
|
| 7 |
|
| 8 |
+
### used -
|
| 9 |
+
```
|
| 10 |
source_model.lm_head.weight.data = source_model.model.embed_tokens.weight.data
|
| 11 |
source_model.lm_head.weight = source_model.model.embed_tokens.weight
|
| 12 |
+
source_model.tie_weights()
|
| 13 |
+
```
|