Update README.md
Browse files
README.md
CHANGED
@@ -16,6 +16,7 @@ tags:
|
|
16 |
|
17 |
## ํ์ธ ํ๋ ๊ฐ์
|
18 |
- ๋์ ๋ชจ๋ธ : meta-llama/Llama-2-7b-hf
|
|
|
19 |
- ํ๋ ๋ฐฉ๋ฒ : PEFT(Parameter Efficient Fine-Tuning) - LoRA(Low-Rank Adaptation of Large Language Models)
|
20 |
- ๊ด๋ จ ๋
ผ๋ฌธ : https://arxiv.org/abs/2106.09685
|
21 |
- Llama2 ๋ชจ๋ธ์ ํ๊ตญ์ด ์ฑ๋ด ๋ฐ์ดํฐ(open korean instructions)์ ์ผ๋ถ๋ฅผ ์ด์ฉํ์ฌ ํ์ธ ํ๋ํ ๋ชจ๋ธ
|
@@ -30,6 +31,7 @@ tags:
|
|
30 |
## ํ์ธ ํ๋ ์์ธ
|
31 |
|
32 |
- ์ํ๊ฐ์ 16์ผ๋ก ์ค์
|
|
|
33 |
```
|
34 |
peft_config = LoraConfig(
|
35 |
lora_alpha=16,
|
|
|
16 |
|
17 |
## ํ์ธ ํ๋ ๊ฐ์
|
18 |
- ๋์ ๋ชจ๋ธ : meta-llama/Llama-2-7b-hf
|
19 |
+
- ๋ฐ์ดํฐ์
: heegyu/open-korean-instructions (10%)
|
20 |
- ํ๋ ๋ฐฉ๋ฒ : PEFT(Parameter Efficient Fine-Tuning) - LoRA(Low-Rank Adaptation of Large Language Models)
|
21 |
- ๊ด๋ จ ๋
ผ๋ฌธ : https://arxiv.org/abs/2106.09685
|
22 |
- Llama2 ๋ชจ๋ธ์ ํ๊ตญ์ด ์ฑ๋ด ๋ฐ์ดํฐ(open korean instructions)์ ์ผ๋ถ๋ฅผ ์ด์ฉํ์ฌ ํ์ธ ํ๋ํ ๋ชจ๋ธ
|
|
|
31 |
## ํ์ธ ํ๋ ์์ธ
|
32 |
|
33 |
- ์ํ๊ฐ์ 16์ผ๋ก ์ค์
|
34 |
+
- r ๊ฐ์ 64๋ก ์ค์ (์ข ํฐ ๋ฏ...ใ
ใ
)
|
35 |
```
|
36 |
peft_config = LoraConfig(
|
37 |
lora_alpha=16,
|