andreaskoepf
commited on
Commit
·
2af2a85
1
Parent(s):
70c654c
Correct link to epfLLM/Megatron-LLM
Browse files
README.md
CHANGED
@@ -20,7 +20,7 @@ on the best human demonstrations collected at [open-assistant.io](https://open-a
|
|
20 |
|
21 |
## Model Details
|
22 |
|
23 |
-
- **Finetuned from:** [meta-llama/Llama-2-70b](https://huggingface.co/meta-llama/Llama-2-70b) via [epfLLM/
|
24 |
- **Model type:** Causal decoder-only transformer language model
|
25 |
- **Language:** English (and limited capabilities in German, Spanish, French, Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish)
|
26 |
- **Weights & Biases training logs:** [Stage 1](https://wandb.ai/open-assistant/public-sft/runs/run45_oasst_pre10_llama2_70b) (1 epoch pretrain-mix, 12k steps), [Stage 2](https://wandb.ai/open-assistant/public-sft/runs/run46_oasst_sft10_llama2_70b) (3 epochs oasst top-1, 519 steps)
|
|
|
20 |
|
21 |
## Model Details
|
22 |
|
23 |
+
- **Finetuned from:** [meta-llama/Llama-2-70b](https://huggingface.co/meta-llama/Llama-2-70b) via [epfLLM/Megatron-LLM](https://github.com/epfLLM/Megatron-LLM)
|
24 |
- **Model type:** Causal decoder-only transformer language model
|
25 |
- **Language:** English (and limited capabilities in German, Spanish, French, Italian, Portuguese, Polish, Dutch, Romanian, Czech, Swedish)
|
26 |
- **Weights & Biases training logs:** [Stage 1](https://wandb.ai/open-assistant/public-sft/runs/run45_oasst_pre10_llama2_70b) (1 epoch pretrain-mix, 12k steps), [Stage 2](https://wandb.ai/open-assistant/public-sft/runs/run46_oasst_sft10_llama2_70b) (3 epochs oasst top-1, 519 steps)
|