leesm commited on
Commit
a3cea18
ยท
verified ยท
1 Parent(s): 691c197

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md CHANGED
@@ -16,6 +16,7 @@ tags:
16
 
17
  ## ํŒŒ์ธ ํŠœ๋‹ ๊ฐœ์š”
18
  - ๋Œ€์ƒ ๋ชจ๋ธ : meta-llama/Llama-2-7b-hf
 
19
  - ํŠœ๋‹ ๋ฐฉ๋ฒ• : PEFT(Parameter Efficient Fine-Tuning) - LoRA(Low-Rank Adaptation of Large Language Models)
20
  - ๊ด€๋ จ ๋…ผ๋ฌธ : https://arxiv.org/abs/2106.09685
21
  - Llama2 ๋ชจ๋ธ์„ ํ•œ๊ตญ์–ด ์ฑ—๋ด‡ ๋ฐ์ดํ„ฐ(open korean instructions)์˜ ์ผ๋ถ€๋ฅผ ์ด์šฉํ•˜์—ฌ ํŒŒ์ธ ํŠœ๋‹ํ•œ ๋ชจ๋ธ
@@ -30,6 +31,7 @@ tags:
30
  ## ํŒŒ์ธ ํŠœ๋‹ ์ƒ์„ธ
31
 
32
  - ์•ŒํŒŒ๊ฐ’์„ 16์œผ๋กœ ์„ค์ •
 
33
  ```
34
  peft_config = LoraConfig(
35
  lora_alpha=16,
 
16
 
17
  ## ํŒŒ์ธ ํŠœ๋‹ ๊ฐœ์š”
18
  - ๋Œ€์ƒ ๋ชจ๋ธ : meta-llama/Llama-2-7b-hf
19
+ - ๋ฐ์ดํ„ฐ์…‹ : heegyu/open-korean-instructions (10%)
20
  - ํŠœ๋‹ ๋ฐฉ๋ฒ• : PEFT(Parameter Efficient Fine-Tuning) - LoRA(Low-Rank Adaptation of Large Language Models)
21
  - ๊ด€๋ จ ๋…ผ๋ฌธ : https://arxiv.org/abs/2106.09685
22
  - Llama2 ๋ชจ๋ธ์„ ํ•œ๊ตญ์–ด ์ฑ—๋ด‡ ๋ฐ์ดํ„ฐ(open korean instructions)์˜ ์ผ๋ถ€๋ฅผ ์ด์šฉํ•˜์—ฌ ํŒŒ์ธ ํŠœ๋‹ํ•œ ๋ชจ๋ธ
 
31
  ## ํŒŒ์ธ ํŠœ๋‹ ์ƒ์„ธ
32
 
33
  - ์•ŒํŒŒ๊ฐ’์„ 16์œผ๋กœ ์„ค์ •
34
+ - r ๊ฐ’์„ 64๋กœ ์„ค์ • (์ข€ ํฐ ๋“ฏ...ใ… ใ… )
35
  ```
36
  peft_config = LoraConfig(
37
  lora_alpha=16,