DiscoLeo 8B: Llama3 for German

DiscoResearch 's Collections

updated May 25

Continued Pretraining on Llama3 8B to improve German linguistic capabilities. A collection of base and fine-tuned models and variants.

DiscoResearch/Llama3-German-8B

Text Generation • Updated Sep 10 • 565 • 38

Note Pretrained model with continued pretraining on 65b tokens of high-quality German texts.
DiscoResearch/Llama3-German-8B-32k

Text Generation • Updated Sep 10 • 179 • 11

Note Same as above but with an additional 100m tokens of pretraining on texts of 32k tokens length and rope_theta=1.5e6 to improve long-context capabilities.
DiscoResearch/Llama3-DiscoLeo-Instruct-8B-v0.1

Text Generation • Updated Sep 10 • 926 • 16

Note DiscoResearch/Llama3_German_8B finetuned on our DiscoLM German Instruction dataset.
DiscoResearch/Llama3-DiscoLeo-Instruct-8B-32k-v0.1

Text Generation • Updated Sep 10 • 179 • 13

Note DiscoResearch/Llama3_German_8B_32k finetuned on our DiscoLM German Instruction dataset.
DiscoResearch/Llama3-DiscoLeo-8B-DARE-Experimental

Text Generation • Updated May 25 • 15

Note Experimental Merge of meta-llama/Meta-Llama-3-8B and DiscoResearch/Llama3_DiscoLeo_Instruct_8B_v0.1 with DARE-TIES and 0.5:0.5 ratio.