leesm commited on
Commit
63b8f71
·
verified ·
1 Parent(s): ae92e99

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -14,13 +14,13 @@ datasets:
14
  - FreedomIntelligence/alpaca-gpt4-korean
15
  ---
16
 
17
- # unsloth/mistral-7b-v0.3 fine tuning after Continued Pretraining
18
  # (TREX-Lab at Seoul Cyber University)
19
 
20
  <!-- Provide a quick summary of what the model is/does. -->
21
 
22
  ## Summary
23
- - Base Model : unsloth/mistral-7b-v0.3
24
  - Dataset : wikimedia/wikipedia(Continued Pretraining), FreedomIntelligence/alpaca-gpt4-korean(FineTuning)
25
  - This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
26
  - Test whether fine tuning of a large language model is possible on A30 GPU*1 (successful)
@@ -29,7 +29,7 @@ datasets:
29
 
30
  - **Developed by:** [TREX-Lab at Seoul Cyber University]
31
  - **Language(s) (NLP):** [Korean]
32
- - **Finetuned from model :** [unsloth/mistral-7b-v0.3]
33
 
34
  ## Continued Pretraining
35
  ```
 
14
  - FreedomIntelligence/alpaca-gpt4-korean
15
  ---
16
 
17
+ # unsloth/Meta-Llama-3.1-8B-bnb-4bit fine tuning after Continued Pretraining
18
  # (TREX-Lab at Seoul Cyber University)
19
 
20
  <!-- Provide a quick summary of what the model is/does. -->
21
 
22
  ## Summary
23
+ - Base Model : unsloth/Meta-Llama-3.1-8B-bnb-4bit
24
  - Dataset : wikimedia/wikipedia(Continued Pretraining), FreedomIntelligence/alpaca-gpt4-korean(FineTuning)
25
  - This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
26
  - Test whether fine tuning of a large language model is possible on A30 GPU*1 (successful)
 
29
 
30
  - **Developed by:** [TREX-Lab at Seoul Cyber University]
31
  - **Language(s) (NLP):** [Korean]
32
+ - **Finetuned from model :** [unsloth/Meta-Llama-3.1-8B-bnb-4bit]
33
 
34
  ## Continued Pretraining
35
  ```