Text Generation
Transformers
Safetensors
llama
text-generation-inference
unsloth
trl
sft
conversational
Inference Endpoints
mpasila commited on
Commit
f6590e6
1 Parent(s): 45ff5fb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -2,6 +2,12 @@
2
  base_model: LumiOpen/Viking-7B
3
  language:
4
  - en
 
 
 
 
 
 
5
  license: apache-2.0
6
  tags:
7
  - text-generation-inference
@@ -10,7 +16,19 @@ tags:
10
  - llama
11
  - trl
12
  - sft
 
 
 
13
  ---
 
 
 
 
 
 
 
 
 
14
 
15
  # Uploaded model
16
 
 
2
  base_model: LumiOpen/Viking-7B
3
  language:
4
  - en
5
+ - fi
6
+ - sv
7
+ - 'no'
8
+ - da
9
+ - is
10
+ - nn
11
  license: apache-2.0
12
  tags:
13
  - text-generation-inference
 
16
  - llama
17
  - trl
18
  - sft
19
+ datasets:
20
+ - Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
21
+ - mpasila/Sonnet3.5-SlimOrcaDedupCleaned-4k-context
22
  ---
23
+ This is the fully trained version (with fixed formatting!!).
24
+
25
+ Dataset used: [Gryphe/Sonnet3.5-SlimOrcaDedupCleaned](https://huggingface.co/datasets/Gryphe/Sonnet3.5-SlimOrcaDedupCleaned) which was further [filtered](https://huggingface.co/datasets/mpasila/Sonnet3.5-SlimOrcaDedupCleaned-4k-context) to remove prompts/examples that are longer than 4076 tokens (removed about 385 examples).
26
+
27
+ Prompt format is: ChatML
28
+
29
+ LoRA: [mpasila/Viking-SlimSonnet-v1-LoRA-7B](https://huggingface.co/mpasila/Viking-SlimSonnet-v1-LoRA-7B)
30
+
31
+ Trained with regular LoRA (not quantized/QLoRA) and LoRA rank was 128 and Alpha set to 32. Trained for 1 epoch using A40 for about 23 hours.
32
 
33
  # Uploaded model
34