Update README.md
Browse files
README.md
CHANGED
@@ -14,13 +14,13 @@ datasets:
|
|
14 |
- FreedomIntelligence/alpaca-gpt4-korean
|
15 |
---
|
16 |
|
17 |
-
# unsloth/
|
18 |
# (TREX-Lab at Seoul Cyber University)
|
19 |
|
20 |
<!-- Provide a quick summary of what the model is/does. -->
|
21 |
|
22 |
## Summary
|
23 |
-
- Base Model : unsloth/
|
24 |
- Dataset : wikimedia/wikipedia(Continued Pretraining), FreedomIntelligence/alpaca-gpt4-korean(FineTuning)
|
25 |
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
26 |
- Test whether fine tuning of a large language model is possible on A30 GPU*1 (successful)
|
@@ -29,7 +29,7 @@ datasets:
|
|
29 |
|
30 |
- **Developed by:** [TREX-Lab at Seoul Cyber University]
|
31 |
- **Language(s) (NLP):** [Korean]
|
32 |
-
- **Finetuned from model :** [unsloth/
|
33 |
|
34 |
## Continued Pretraining
|
35 |
```
|
|
|
14 |
- FreedomIntelligence/alpaca-gpt4-korean
|
15 |
---
|
16 |
|
17 |
+
# unsloth/Meta-Llama-3.1-8B-bnb-4bit fine tuning after Continued Pretraining
|
18 |
# (TREX-Lab at Seoul Cyber University)
|
19 |
|
20 |
<!-- Provide a quick summary of what the model is/does. -->
|
21 |
|
22 |
## Summary
|
23 |
+
- Base Model : unsloth/Meta-Llama-3.1-8B-bnb-4bit
|
24 |
- Dataset : wikimedia/wikipedia(Continued Pretraining), FreedomIntelligence/alpaca-gpt4-korean(FineTuning)
|
25 |
- This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
26 |
- Test whether fine tuning of a large language model is possible on A30 GPU*1 (successful)
|
|
|
29 |
|
30 |
- **Developed by:** [TREX-Lab at Seoul Cyber University]
|
31 |
- **Language(s) (NLP):** [Korean]
|
32 |
+
- **Finetuned from model :** [unsloth/Meta-Llama-3.1-8B-bnb-4bit]
|
33 |
|
34 |
## Continued Pretraining
|
35 |
```
|