Wants to know how to deploy model and try it for my own
#4
by
LeonBlue
- opened
README.md
CHANGED
|
@@ -2,22 +2,14 @@
|
|
| 2 |
license: mit
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
-
metrics:
|
| 6 |
-
- accuracy
|
| 7 |
-
library_name: transformers
|
| 8 |
---
|
| 9 |
-
|
| 10 |
-
# 24/04/05 update
|
| 11 |
-
We introduce [Moreh AI Model Hub with AMD GPU](https://model-hub.moreh.io/), an ai model host platform powered by AMD MI250 GPUs.
|
| 12 |
-
You can now test live-inference of this model at Moreh AI Model Hub.
|
| 13 |
-
|
| 14 |
# **Introduction**
|
| 15 |
MoMo-72B-lora-1.8.7-DPO is trained via Direct Preference Optimization([DPO](https://arxiv.org/abs/2305.18290)) from [MoMo-72B-LoRA-V1.4](https://huggingface.co/moreh/MoMo-72B-LoRA-V1.4) as its base model, with several optimizations in hyperparameters.
|
| 16 |
[MoMo-72B-LoRA-V1.4](https://huggingface.co/moreh/MoMo-72B-LoRA-V1.4) is trained via Supervised Fine-Tuning (SFT) using [LoRA](https://arxiv.org/abs/2106.09685), with the QWEN-72B model as its base-model.
|
| 17 |
Note that we did not exploit any form of weight merge.
|
| 18 |
For leaderboard submission, the trained weight is realigned for compatibility with llama.
|
| 19 |
MoMo-72B is trained using **[Moreh](https://moreh.io/)**'s [MoAI platform](https://moreh.io/product), which simplifies the training of large-scale models, and AMD's MI250 GPU.
|
| 20 |
-
|
| 21 |
|
| 22 |
## Details
|
| 23 |
### Used Librarys
|
|
|
|
| 2 |
license: mit
|
| 3 |
language:
|
| 4 |
- en
|
|
|
|
|
|
|
|
|
|
| 5 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
# **Introduction**
|
| 7 |
MoMo-72B-lora-1.8.7-DPO is trained via Direct Preference Optimization([DPO](https://arxiv.org/abs/2305.18290)) from [MoMo-72B-LoRA-V1.4](https://huggingface.co/moreh/MoMo-72B-LoRA-V1.4) as its base model, with several optimizations in hyperparameters.
|
| 8 |
[MoMo-72B-LoRA-V1.4](https://huggingface.co/moreh/MoMo-72B-LoRA-V1.4) is trained via Supervised Fine-Tuning (SFT) using [LoRA](https://arxiv.org/abs/2106.09685), with the QWEN-72B model as its base-model.
|
| 9 |
Note that we did not exploit any form of weight merge.
|
| 10 |
For leaderboard submission, the trained weight is realigned for compatibility with llama.
|
| 11 |
MoMo-72B is trained using **[Moreh](https://moreh.io/)**'s [MoAI platform](https://moreh.io/product), which simplifies the training of large-scale models, and AMD's MI250 GPU.
|
| 12 |
+
|
| 13 |
|
| 14 |
## Details
|
| 15 |
### Used Librarys
|