beomi commited on
Commit
c4be33d
1 Parent(s): 4b1aa4a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -39,6 +39,8 @@ Solar-Ko-Recovery is an auto-regressive language model that leverages an optimiz
39
  |---|---|---|---|---|---|---|
40
  |Solar-Ko-Recovery|*A curated mix of Korean+English Corpora*|10.8B|4k|O|>30B*|5e<sup>-5</sup>|
41
 
 
 
42
  **Vocab Expansion**
43
 
44
  Vocab expansion is conducted on edited [upstage/solar-1-mini-tokenizer](https://huggingface.co/upstage/solar-1-mini-tokenizer), which is superset of Solar tokenizer.
 
39
  |---|---|---|---|---|---|---|
40
  |Solar-Ko-Recovery|*A curated mix of Korean+English Corpora*|10.8B|4k|O|>30B*|5e<sup>-5</sup>|
41
 
42
+ > NOTE: Only Embedding layer and LM Head layer are trained.
43
+
44
  **Vocab Expansion**
45
 
46
  Vocab expansion is conducted on edited [upstage/solar-1-mini-tokenizer](https://huggingface.co/upstage/solar-1-mini-tokenizer), which is superset of Solar tokenizer.