Hugging Face's logo

RichardErkhov
/
sbintuitions_-_sarashina2-13b-gguf

GGUF
Inference Endpoints
Model card Files Community
RichardErkhov commited on
Commit
6b44119
·
verified ·
1 Parent(s): 3e02621

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +114 -0
README.md ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ sarashina2-13b - GGUF
11
+ - Model creator: https://huggingface.co/sbintuitions/
12
+ - Original model: https://huggingface.co/sbintuitions/sarashina2-13b/
13
+
14
+
15
+ | Name | Quant method | Size |
16
+ | ---- | ---- | ---- |
17
+ | [sarashina2-13b.Q2_K.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q2_K.gguf) | Q2_K | 4.91GB |
18
+ | [sarashina2-13b.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.IQ3_XS.gguf) | IQ3_XS | 5.41GB |
19
+ | [sarashina2-13b.IQ3_S.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.IQ3_S.gguf) | IQ3_S | 5.69GB |
20
+ | [sarashina2-13b.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q3_K_S.gguf) | Q3_K_S | 5.69GB |
21
+ | [sarashina2-13b.IQ3_M.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.IQ3_M.gguf) | IQ3_M | 5.99GB |
22
+ | [sarashina2-13b.Q3_K.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q3_K.gguf) | Q3_K | 6.32GB |
23
+ | [sarashina2-13b.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q3_K_M.gguf) | Q3_K_M | 6.32GB |
24
+ | [sarashina2-13b.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q3_K_L.gguf) | Q3_K_L | 6.87GB |
25
+ | [sarashina2-13b.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.IQ4_XS.gguf) | IQ4_XS | 6.99GB |
26
+ | [sarashina2-13b.Q4_0.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q4_0.gguf) | Q4_0 | 7.33GB |
27
+ | [sarashina2-13b.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.IQ4_NL.gguf) | IQ4_NL | 7.37GB |
28
+ | [sarashina2-13b.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q4_K_S.gguf) | Q4_K_S | 7.38GB |
29
+ | [sarashina2-13b.Q4_K.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q4_K.gguf) | Q4_K | 7.79GB |
30
+ | [sarashina2-13b.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q4_K_M.gguf) | Q4_K_M | 7.79GB |
31
+ | [sarashina2-13b.Q4_1.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q4_1.gguf) | Q4_1 | 8.09GB |
32
+ | [sarashina2-13b.Q5_0.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q5_0.gguf) | Q5_0 | 8.86GB |
33
+ | [sarashina2-13b.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q5_K_S.gguf) | Q5_K_S | 8.86GB |
34
+ | [sarashina2-13b.Q5_K.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q5_K.gguf) | Q5_K | 9.1GB |
35
+ | [sarashina2-13b.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q5_K_M.gguf) | Q5_K_M | 9.1GB |
36
+ | [sarashina2-13b.Q5_1.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q5_1.gguf) | Q5_1 | 9.63GB |
37
+ | [sarashina2-13b.Q6_K.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q6_K.gguf) | Q6_K | 10.5GB |
38
+ | [sarashina2-13b.Q8_0.gguf](https://huggingface.co/RichardErkhov/sbintuitions_-_sarashina2-13b-gguf/blob/main/sarashina2-13b.Q8_0.gguf) | Q8_0 | 13.6GB |
39
+
40
+
41
+
42
+
43
+ Original model description:
44
+ ---
45
+ license: mit
46
+ language:
47
+ - ja
48
+ - en
49
+ ---
50
+
51
+ # Sarashina2-13B
52
+
53
+ This repository provides large language models trained by [SB Intuitions](https://www.sbintuitions.co.jp/).
54
+
55
+
56
+ ## How to use
57
+
58
+
59
+ ```python
60
+ import torch
61
+ from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline, set_seed
62
+
63
+ model = AutoModelForCausalLM.from_pretrained("sbintuitions/sarashina2-13b", torch_dtype=torch.bfloat16, device_map="auto")
64
+ tokenizer = AutoTokenizer.from_pretrained("sbintuitions/sarashina2-13b")
65
+ # If you want to use slow tokenizer
66
+ # tokenizer = AutoTokenizer.from_pretrained("sbintuitions/sarashina2-13b", use_fast=False）
67
+ generator = pipeline("text-generation", model=model, tokenizer=tokenizer)
68
+ set_seed(123)
69
+
70
+ text = generator(
71
+ "おはようございます、今日の天気は",
72
+ max_length=30,
73
+ do_sample=True,
74
+ pad_token_id=tokenizer.pad_token_id,
75
+ num_return_sequences=3,
76
+ )
77
+
78
+ for t in text:
79
+ print(t)
80
+
81
+ ```
82
+
83
+ ## Configuration
84
+
85
+ | Parameters | Vocab size | Training tokens | Architecture | Position type | Layers | Hidden dim | Attention heads |
86
+ | :-----: | :-----------: | :-------------: | :------------ | :-----------: | :----: | :--------: | :-------------: |
87
+ | [7B](https://huggingface.co/sbintuitions/sarashina2-7b) | 102400 | 2.1T | Llama2 | RoPE | 32 | 4096 | 32 |
88
+ | [13B](https://huggingface.co/sbintuitions/sarashina2-13b) | 102400 | 2.1T | Llama2 | RoPE | 40 | 5120 | 40 |
89
+ | [70B](https://huggingface.co/sbintuitions/sarashina2-70b) | 102400 | 2.1T | Llama2 | RoPE | 80 | 8192 | 64 |
90
+
91
+ ## Training Corpus
92
+
93
+ For our Japanese training data, we used a Japanese portion of the [Common Crawl corpus](https://commoncrawl.org/), which is the largest Web corpus, as our training dataset.
94
+ To clean the training corpus, we used [CCNet](https://github.com/facebookresearch/cc_net) and [HojiChar](https://github.com/HojiChar/HojiChar).
95
+ After cleaning, our Japanese training data contains about 1T tokens.
96
+
97
+ For our English training data, we extracted English documents from [SlimPajama](https://huggingface.co/datasets/cerebras/SlimPajama-627B) but we removed books3 corpus due to copyright infringement.
98
+
99
+ ## Tokenization
100
+
101
+ We use a [sentencepiece](https://github.com/google/sentencepiece) tokenizer with a unigram language model and byte-fallback.
102
+ We do not apply pre-tokenization with Japanese tokenizer.
103
+ Thus, a user may directly feed raw sentences into the tokenizer.
104
+
105
+
106
+ ## Ethical Considerations and Limitations
107
+ Sarashina2 has not been tuned to follow an instruction yet.
108
+ Therefore, sarashina2 might generate some meaningless sequences, some inaccurate instances or biased/objectionable outputs.
109
+ Before using sarashina2, we would like developers to tune models based on human preferences and safety considerations.
110
+
111
+ ## License
112
+
113
+ [MIT License](https://huggingface.co/sbintuitions/sarashina2-7b/blob/main/LICENSE)
114
+