Update README.md
Browse files
README.md
CHANGED
@@ -9,14 +9,14 @@ tags:
|
|
9 |
- unsloth
|
10 |
- qwen2
|
11 |
- trl
|
|
|
|
|
12 |
---
|
13 |
|
14 |
-
#
|
15 |
|
16 |
- **Developed by:** Nelathan
|
17 |
- **License:** apache-2.0
|
18 |
- **Finetuned from model :** unsloth/Qwen2.5-3B-bnb-4bit
|
19 |
|
20 |
-
|
21 |
-
|
22 |
-
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
|
|
9 |
- unsloth
|
10 |
- qwen2
|
11 |
- trl
|
12 |
+
datasets:
|
13 |
+
- roneneldan/TinyStories
|
14 |
---
|
15 |
|
16 |
+
# 5k Tinystory QLora(rank=128, alpha=32, lr=5e-5)
|
17 |
|
18 |
- **Developed by:** Nelathan
|
19 |
- **License:** apache-2.0
|
20 |
- **Finetuned from model :** unsloth/Qwen2.5-3B-bnb-4bit
|
21 |
|
22 |
+
Seems to be very overfit but coherent
|
|
|
|