Hugging Face's logo

MgGPT
/
MgGPT-13B

Safetensors
Arabic
llama
Model card Files Community
Jianqing Zhu commited on
Commit
c9cb19a
·
1 Parent(s): 74530e8

Initial commit

Browse files
Files changed (19) hide show
  1. README.md +83 -0
  2. config.json +30 -0
  3. generation_config.json +8 -0
  4. model-00001-of-00011.safetensors +3 -0
  5. model-00002-of-00011.safetensors +3 -0
  6. model-00003-of-00011.safetensors +3 -0
  7. model-00004-of-00011.safetensors +3 -0
  8. model-00005-of-00011.safetensors +3 -0
  9. model-00006-of-00011.safetensors +3 -0
  10. model-00007-of-00011.safetensors +3 -0
  11. model-00008-of-00011.safetensors +3 -0
  12. model-00009-of-00011.safetensors +3 -0
  13. model-00010-of-00011.safetensors +3 -0
  14. model-00011-of-00011.safetensors +3 -0
  15. model.safetensors.index.json +370 -0
  16. special_tokens_map.json +23 -0
  17. tokenizer.json +0 -0
  18. tokenizer.model +3 -0
  19. tokenizer_config.json +42 -0
README.md ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - ar
5
+ - zh
6
+ - en
7
+ ---
8
+
9
+ # <b>AceGPT</b>
10
+
11
+ AceGPT is a fully fine-tuned generative text model collection based on LlaMA2, particularly in the
12
+ Arabic language domain. This is the repository for the version 1.5 of 13B pre-trained model.
13
+
14
+ ---
15
+ ## Model Details
16
+ We have released the AceGPT family of large language models, which is a collection of fully fine-tuned generative text models based on LlaMA2, ranging from 7B to 13B parameters. Our models include two main categories: AceGPT and AceGPT-chat. AceGPT-chat is an optimized version specifically designed for dialogue applications. It is worth mentioning that our models have demonstrated superior performance compared to all currently available open-source Arabic dialogue models in multiple benchmark tests. Furthermore, in our human evaluations, our models have shown comparable satisfaction levels to some closed-source models, such as ChatGPT, in the Arabic language.
17
+ ## Model Developers
18
+ We are from the King Abdullah University of Science and Technology (KAUST), the Chinese University of Hong Kong, Shenzhen (CUHKSZ), the Shenzhen Research Institute of Big Data (SRIBD), and King AbdulAziz University (KAU).
19
+ ## Variations
20
+ AceGPT families come in a range of parameter sizes —— 7B and 13B, each size of model has a base category and a -chat category.
21
+ ## Paper
22
+ The paper can be accessed at [link](https://huggingface.co/FreedomIntelligence/AceGPT-v1.5-13B-Chat/blob/main/Second_Language_(Arabic)_Acquisition_of_LLMs_via_Progressive_Vocabulary_Expansion.pdf).
23
+ ## Input
24
+ Models input text only.
25
+ ## Output
26
+ Models output text only.
27
+ ## Model Evaluation Results
28
+
29
+ Benchmark evaluation on [Arabic MMLU](https://github.com/FreedomIntelligence/AceGPT) are conducted using accuracy scores as metrics, following the evaluation framework available at https://github.com/FreedomIntelligence/AceGPT/tree/main.
30
+ | | STEM | Humanities | Social Sciences | Others | Average |
31
+ |------------------|------|------|------|------|------|
32
+ | Bloomz-7B-base | 33.35 | 29.29 | 37.58 | 34.53 | 33.69 |
33
+ | LLaMA2-7B-base | 30.30 | 29.33 | 27.46 | 30.78 | 29.37 |
34
+ | AceGPT-7B-base | 29.73 | 30.95 | 33.45 | 34.42 | 32.14 |
35
+ | AceGPT-v1.5-7B-base | 33.03 | 32.08 | 35.39 | 35.59 | 34.03 |
36
+ | LLaMA2-13B-base | 32.94 | 32.30 | 33.42 | 37.27 | 33.76 |
37
+ | Jais-13B-base | 30.51 | 31.25 | 33.74 | 33.42 | 33.76 |
38
+ | AceGPT-13B-base | 36.60 | 38.74 | 43.76 | <u>42.72</u> | 40.45 |
39
+ | AceGPT-v1.5-13B-base | <u>36.13</u> | <u>40.07</u> | <u>45.43</u> | 42.17 | <u>40.95</u> |
40
+ | Jais-30B-v1-base | 32.67 | 30.67 | 42.13 | 39.60 | 36.27 |
41
+ | ChatGPT 3.5 Turbo | **43.38** | **44.12** | **55.57** | **53.21** | **49.07** |
42
+
43
+
44
+ Benchmark evaluation on [ArabicMMLU]((https://github.com/mbzuai-nlp/ArabicMMLU)), and assessed based on its source settings.
45
+ | | STEM | Social Sciences | Humanities | Arabic Language | Other | Average |
46
+ |------------------|------|------|------|------|------|------|
47
+ | Bloomz-7B-base | - | - | - | - | - | - |
48
+ | LLaMA2-7B-base | 33.7 | 32.8 | 33.5 | 28.4 | 36.7 | 33.4 |
49
+ | AceGPT-7B-base | 35.4 | 35.9 | 36.2 | 31.1 | 41.7 | 36.3 |
50
+ | AceGPT-v1.5-7B-base | 36.7 | 36.5 | 34.1 | 30.0 | 41.2 | 37.0 |
51
+ | LLaMA2-13B-base | 32.9 | 35.0 | 37.8 | 35.8 | 39.3 | 36.1 |
52
+ | Jais-13B-base | 30.3 | 31.4 | 33.6 | 28.1 | 36.3 | 32.2 |
53
+ | AceGPT-13B-base | <u>42.7</u> | 45.5 | 48.3 | 42.4 | 50.7 | 46.1 |
54
+ | AceGPT-v1.5-13B-base | 42.4 | <u>45.7</u> | 48.4 | <u>46.3</u> | <u>52.5</u> | <u>47.6</u> |
55
+ | Jais-30B-v1-base | 39.5 | 45.6 | <u>50.5</u> | 34.6 | 49.1 | 44.8 |
56
+ | ChatGPT 3.5 Turbo | **53.8** | **57.0** | **57.5** | **57.6** | **63.8** | **57.7** |
57
+
58
+
59
+ ## Samples
60
+ #### Sample1(abstract_algebra)
61
+ * <b>input:</b>
62
+ "فيما يلي أسئلة الاختيار من متعدد (مع الإجابات) حول جبر تجريدي\n\nسؤال: العثور على جميع قيم c في Z_3 بحيث يكون Z_3 [x]/(x^2+c) حقلًا.\nA. 0\nB. 1\nC. 2\nD. 3\nإجابة: B\n\nسؤال: البيان رقم 1 | إذا كان aH عنصرًا في مجموعة العوامل ، فإن | aH | يقسم | a |. البيان رقم 2 | إذا كانت H و K مجموعات فرعية لـ G ، فإن HK مجموعة فرعية لـ G.\nA. صحيح ، صحيح\nB. خطأ ، خطأ\nC. صحيح ، خطأ\nD. خطأ ، صحيح\nإجابة: B\n\nسؤال: العبارة 1 | كل عنصر من مجموعة يولد مجموعة دورية من المجموعة. العبارة 2 | المجموعة المتناظرة S_10 لديها 10 عناصر.\nA. صحيح، صحيح\nB. خطأ، خطأ\nC. صحيح، خطأ\nD. خطأ، صحيح\nإجابة: C\n\nسؤال: البيان 1| كل وظيفة من مجموعة محدودة على نفسها يجب أن تكون واحدة لكل مجموعة. البيان 2 | كل فرع فرعي لمجموعة أبيلية هو أبيلي.\nA. صحيح, صحيح\nB. خاطئ, خاطئ\nC. صحيح, خاطئ\nD. خاطئ, صحيح\nإجابة: A\n\nسؤال: اعثر على خاصية الحل��ة 2Z.\nA. 0\nB. 3\nC. 12\nD. 30\nإجابة: A\n\nسؤال: ما هو الدرجة للامتداد الميداني الناتج من Q(sqrt(2), sqrt(3), sqrt(18)) على Q؟\nA. 0\nB. 4\nC. 2\nD. 6\nإجابة:"
63
+
64
+ * <b>output:</b>
65
+ "B\n\nسؤال: ما هو عدد العناصر"
66
+
67
+ #### Sample2(business_ethics)
68
+ * <b>input:</b>
69
+ "فيما يلي أسئلة الاختيار من متعدد (مع الإجابات) حول أخلاقيات الأعمال\n\nسؤال: ما هي الحجج الأخلاقية المتعلقة بالمسؤولية الاجتماعية للشركات؟\nA. التكاليف الخارجية، القوة، الاستقلالية\nB. الإعلام، الموارد الضعيفة، التبادل التعاوني\nC. الإعلام، القوة، الاستقلالية\nD. التكاليف الخارجية، القوة، التبادل التعاوني\nإجابة: D\n\nسؤال: _______ هو المحاولة المباشرة لإدارة القضايا الأخلاقية أو المشاكل، سواء بشكل رسمي أو غير رسمي، من خلال سياسات وممارسات وبرامج محددة.\nA. المسؤولية الاجتماعية للشركات\nB. إدارة الأخلاقيات العملية\nC. الاستدامة\nD. إدارة البيئة\nإجابة: B\n\nسؤال: لضمان استقلال أعضاء مجلس الإدارة غير التنفيذية ، هناك عدد من الخطوات التي يمكن اتخاذها ، والتي تشمل اختيار الغير التنفيذيين من _______ الشركة ، وتعيينهم لمدة _________ ، وكذلك تعيينهم _________.\nA. خارج الشركة ، محدودة ، بشكل مستقل\nB. من الداخل ، محدودة ، بشكل متقطع\nC. خارج الشركة ، غير محدودة ، بشكل متقطع\nD. من الداخل ، غير محدودة ، بشكل مستقل\nإجابة: A\n\nسؤال: ما هي الأساليب التي يمكن للمدير الأمني الذي يسعى لتحقيق أهدافه الاختيار بينها؟\nA. العمل المباشر الغير عنيف ، العمل المباشر العنيف ، العمل غير المباشر ، الحملة الدعائية\nB. العمل غير المباشر ، العمل الأوتيل ، العمل المباشر الغير عنيف ، الحملة الإعلامية\nC. العمل غير المباشر ، العمل المباشر العنيف ، العمل المباشر غير العنيف المباشر ، الحملة الدعائية\nD. العمل المباشر الغير عنيف ، العمل الأوتيل ، العمل غير المباشر ، الحملة الإعلامية\nإجابة: C\n\nسؤال: على عكس _______ ، تهدف _______ إلى مكافأة السلوك الإيجابي للشركات. تم تعزيز نجاح مثل هذه الحملات من خلال استخدام ___________, الذي يتيح للحملات تيسير تحقيق الشركة لــ _________ .\nA. الحملات الاستهلاكية، الحملات الاستهلاكية العامة، تكنولوجيا سلسلة الكتل، التبرعات الخيرية\nB. الحملات التحفيزية، الحملات الاستهلاكية العامة، التكنولوجيا الرقمية، زيادة المبيعات\nC. الحملات الاستهلاكية، الحملات الشرائية، تكنولوجيا سلسلة الكتل، التبرعات الخيرية\nD. المقاطعات، الحملات التحفيزية، الحملات الرقمية، زيادة المبيعات\nإجابة: D\n\nسؤال: تُصبح _______ مثل البيتكوين أكثر انتشارًا وتحمل مجموعة كبيرة من الآثار الأخلاقية المرتبطة بها، على سبيل المثال، إنها _______ وأكثر _______. ومع ذلك، تم استخدامها أيضًا للمشاركة في _______.\nA. العملات الرقمية، مكلفة، آمنة، جرائم مالية\nB. العملات التقليدية، رخيصة، غير آمنة، العطاء الخيري\nC. العملات الرقمية، رخيصة، آمنة، جرائم مالية\nD. العملات التقليدية، مكلفة، غير آمنة، العطاء الخيري\nإجابة:"
70
+
71
+ * <b>output:</b>
72
+ "A\n\nسؤال: _______ هو"
73
+
74
+
75
+ # Reference
76
+ ```
77
+ @article{zhu2024second,
78
+ title={Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion},
79
+ author={Zhu, Jianqing and Huang, Huang and Lin, Zhihang and Liang, Juhao and Tang, Zhengyang and Almubarak, Khalid and Alharthi, Mosen and An, Bang and He, Juncai and Wu, Xiangbo and Yu, Fei and Chen, Junying and Ma, Zhuoheng and Du, Yuhao and Hu, Yan and Zhang, He and Alghamdi, Emad A. and Zhang, Lian and Sun, Ruoyu and Li, Haizhou and Wang, Benyou and Xu, Jinchao},
80
+ journal={},
81
+ year={2024}
82
+ }
83
+ ```
config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "AceGPT-v1.5-13B_1",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 1,
9
+ "eos_token_id": 2,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 5120,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 13824,
14
+ "max_length": 4096,
15
+ "max_position_embeddings": 4096,
16
+ "model_type": "llama",
17
+ "num_attention_heads": 40,
18
+ "num_hidden_layers": 40,
19
+ "num_key_value_heads": 40,
20
+ "pad_token_id": 0,
21
+ "pretraining_tp": 1,
22
+ "rms_norm_eps": 1e-05,
23
+ "rope_scaling": null,
24
+ "rope_theta": 10000.0,
25
+ "tie_word_embeddings": false,
26
+ "torch_dtype": "float32",
27
+ "transformers_version": "4.38.1",
28
+ "use_cache": true,
29
+ "vocab_size": 44800
30
+ }
generation_config.json ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 1,
4
+ "eos_token_id": 2,
5
+ "max_length": 4096,
6
+ "pad_token_id": 0,
7
+ "transformers_version": "4.38.1"
8
+ }
model-00001-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1803844f69256dafa51bf93729d0c78796658414757385a2653d7c928a9ae078
3
+ size 4933676424
model-00002-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b625ace0aa504a734b66059f6f6d1d236e26f4d82ac0192bdb81c1a65ca0813f
3
+ size 4970418112
model-00003-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3c11faa2351ba11da5f98e0905d732dc3fa19766024deb9c36c291131ffbe71c
3
+ size 4970418120
model-00004-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:fe7407ac9c53ddf9a078648e27df6d6901765e8698d80dde5239b685d66160e1
3
+ size 4792119040
model-00005-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:291c8bb5475c6a04d12f6e895bab25d98efd5518cca792c8b284d20807384bba
3
+ size 4792160232
model-00006-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:92bd739fb98fc30cb6cb33e4183499a94710a4abf209bb5f7a501f4f9bb3c802
3
+ size 4792160224
model-00007-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:28ffd541d8a1be910b39fa37b4248b936b8f54fc1938727e7bafb255951c0b77
3
+ size 4970418144
model-00008-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2778b7fcae680c47a92ce99fac084ba79bdd0380d077baf7d14235f4594004d
3
+ size 4970418144
model-00009-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7394b54397d5dbcb9abf09d038e239ee7a0e3d31463c08357430201e4c0a4159
3
+ size 4970418144
model-00010-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dbd6691715a92e25116c803b56bee871d0686d8676daa99931b7cc86ede57c90
3
+ size 4970418144
model-00011-of-00011.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0f9632728b0b8b5e85b7cf20c9c96f12e45ecd6c4c24806f93f954d217ed59ca
3
+ size 3455162616
model.safetensors.index.json ADDED
@@ -0,0 +1,370 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 52587745280
4
+ },
5
+ "weight_map": {
6
+ "lm_head.weight": "model-00011-of-00011.safetensors",
7
+ "model.embed_tokens.weight": "model-00001-of-00011.safetensors",
8
+ "model.layers.0.input_layernorm.weight": "model-00001-of-00011.safetensors",
9
+ "model.layers.0.mlp.down_proj.weight": "model-00001-of-00011.safetensors",
10
+ "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00011.safetensors",
11
+ "model.layers.0.mlp.up_proj.weight": "model-00001-of-00011.safetensors",
12
+ "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00011.safetensors",
13
+ "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00011.safetensors",
14
+ "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00011.safetensors",
15
+ "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00011.safetensors",
16
+ "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00011.safetensors",
17
+ "model.layers.1.input_layernorm.weight": "model-00001-of-00011.safetensors",
18
+ "model.layers.1.mlp.down_proj.weight": "model-00001-of-00011.safetensors",
19
+ "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00011.safetensors",
20
+ "model.layers.1.mlp.up_proj.weight": "model-00001-of-00011.safetensors",
21
+ "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00011.safetensors",
22
+ "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00011.safetensors",
23
+ "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00011.safetensors",
24
+ "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00011.safetensors",
25
+ "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00011.safetensors",
26
+ "model.layers.10.input_layernorm.weight": "model-00003-of-00011.safetensors",
27
+ "model.layers.10.mlp.down_proj.weight": "model-00003-of-00011.safetensors",
28
+ "model.layers.10.mlp.gate_proj.weight": "model-00003-of-00011.safetensors",
29
+ "model.layers.10.mlp.up_proj.weight": "model-00003-of-00011.safetensors",
30
+ "model.layers.10.post_attention_layernorm.weight": "model-00003-of-00011.safetensors",
31
+ "model.layers.10.self_attn.k_proj.weight": "model-00003-of-00011.safetensors",
32
+ "model.layers.10.self_attn.o_proj.weight": "model-00003-of-00011.safetensors",
33
+ "model.layers.10.self_attn.q_proj.weight": "model-00003-of-00011.safetensors",
34
+ "model.layers.10.self_attn.v_proj.weight": "model-00003-of-00011.safetensors",
35
+ "model.layers.11.input_layernorm.weight": "model-00004-of-00011.safetensors",
36
+ "model.layers.11.mlp.down_proj.weight": "model-00004-of-00011.safetensors",
37
+ "model.layers.11.mlp.gate_proj.weight": "model-00004-of-00011.safetensors",
38
+ "model.layers.11.mlp.up_proj.weight": "model-00004-of-00011.safetensors",
39
+ "model.layers.11.post_attention_layernorm.weight": "model-00004-of-00011.safetensors",
40
+ "model.layers.11.self_attn.k_proj.weight": "model-00004-of-00011.safetensors",
41
+ "model.layers.11.self_attn.o_proj.weight": "model-00004-of-00011.safetensors",
42
+ "model.layers.11.self_attn.q_proj.weight": "model-00004-of-00011.safetensors",
43
+ "model.layers.11.self_attn.v_proj.weight": "model-00004-of-00011.safetensors",
44
+ "model.layers.12.input_layernorm.weight": "model-00004-of-00011.safetensors",
45
+ "model.layers.12.mlp.down_proj.weight": "model-00004-of-00011.safetensors",
46
+ "model.layers.12.mlp.gate_proj.weight": "model-00004-of-00011.safetensors",
47
+ "model.layers.12.mlp.up_proj.weight": "model-00004-of-00011.safetensors",
48
+ "model.layers.12.post_attention_layernorm.weight": "model-00004-of-00011.safetensors",
49
+ "model.layers.12.self_attn.k_proj.weight": "model-00004-of-00011.safetensors",
50
+ "model.layers.12.self_attn.o_proj.weight": "model-00004-of-00011.safetensors",
51
+ "model.layers.12.self_attn.q_proj.weight": "model-00004-of-00011.safetensors",
52
+ "model.layers.12.self_attn.v_proj.weight": "model-00004-of-00011.safetensors",
53
+ "model.layers.13.input_layernorm.weight": "model-00004-of-00011.safetensors",
54
+ "model.layers.13.mlp.down_proj.weight": "model-00004-of-00011.safetensors",
55
+ "model.layers.13.mlp.gate_proj.weight": "model-00004-of-00011.safetensors",
56
+ "model.layers.13.mlp.up_proj.weight": "model-00004-of-00011.safetensors",
57
+ "model.layers.13.post_attention_layernorm.weight": "model-00004-of-00011.safetensors",
58
+ "model.layers.13.self_attn.k_proj.weight": "model-00004-of-00011.safetensors",
59
+ "model.layers.13.self_attn.o_proj.weight": "model-00004-of-00011.safetensors",
60
+ "model.layers.13.self_attn.q_proj.weight": "model-00004-of-00011.safetensors",
61
+ "model.layers.13.self_attn.v_proj.weight": "model-00004-of-00011.safetensors",
62
+ "model.layers.14.input_layernorm.weight": "model-00005-of-00011.safetensors",
63
+ "model.layers.14.mlp.down_proj.weight": "model-00005-of-00011.safetensors",
64
+ "model.layers.14.mlp.gate_proj.weight": "model-00004-of-00011.safetensors",
65
+ "model.layers.14.mlp.up_proj.weight": "model-00004-of-00011.safetensors",
66
+ "model.layers.14.post_attention_layernorm.weight": "model-00005-of-00011.safetensors",
67
+ "model.layers.14.self_attn.k_proj.weight": "model-00004-of-00011.safetensors",
68
+ "model.layers.14.self_attn.o_proj.weight": "model-00004-of-00011.safetensors",
69
+ "model.layers.14.self_attn.q_proj.weight": "model-00004-of-00011.safetensors",
70
+ "model.layers.14.self_attn.v_proj.weight": "model-00004-of-00011.safetensors",
71
+ "model.layers.15.input_layernorm.weight": "model-00005-of-00011.safetensors",
72
+ "model.layers.15.mlp.down_proj.weight": "model-00005-of-00011.safetensors",
73
+ "model.layers.15.mlp.gate_proj.weight": "model-00005-of-00011.safetensors",
74
+ "model.layers.15.mlp.up_proj.weight": "model-00005-of-00011.safetensors",
75
+ "model.layers.15.post_attention_layernorm.weight": "model-00005-of-00011.safetensors",
76
+ "model.layers.15.self_attn.k_proj.weight": "model-00005-of-00011.safetensors",
77
+ "model.layers.15.self_attn.o_proj.weight": "model-00005-of-00011.safetensors",
78
+ "model.layers.15.self_attn.q_proj.weight": "model-00005-of-00011.safetensors",
79
+ "model.layers.15.self_attn.v_proj.weight": "model-00005-of-00011.safetensors",
80
+ "model.layers.16.input_layernorm.weight": "model-00005-of-00011.safetensors",
81
+ "model.layers.16.mlp.down_proj.weight": "model-00005-of-00011.safetensors",
82
+ "model.layers.16.mlp.gate_proj.weight": "model-00005-of-00011.safetensors",
83
+ "model.layers.16.mlp.up_proj.weight": "model-00005-of-00011.safetensors",
84
+ "model.layers.16.post_attention_layernorm.weight": "model-00005-of-00011.safetensors",
85
+ "model.layers.16.self_attn.k_proj.weight": "model-00005-of-00011.safetensors",
86
+ "model.layers.16.self_attn.o_proj.weight": "model-00005-of-00011.safetensors",
87
+ "model.layers.16.self_attn.q_proj.weight": "model-00005-of-00011.safetensors",
88
+ "model.layers.16.self_attn.v_proj.weight": "model-00005-of-00011.safetensors",
89
+ "model.layers.17.input_layernorm.weight": "model-00005-of-00011.safetensors",
90
+ "model.layers.17.mlp.down_proj.weight": "model-00005-of-00011.safetensors",
91
+ "model.layers.17.mlp.gate_proj.weight": "model-00005-of-00011.safetensors",
92
+ "model.layers.17.mlp.up_proj.weight": "model-00005-of-00011.safetensors",
93
+ "model.layers.17.post_attention_layernorm.weight": "model-00005-of-00011.safetensors",
94
+ "model.layers.17.self_attn.k_proj.weight": "model-00005-of-00011.safetensors",
95
+ "model.layers.17.self_attn.o_proj.weight": "model-00005-of-00011.safetensors",
96
+ "model.layers.17.self_attn.q_proj.weight": "model-00005-of-00011.safetensors",
97
+ "model.layers.17.self_attn.v_proj.weight": "model-00005-of-00011.safetensors",
98
+ "model.layers.18.input_layernorm.weight": "model-00006-of-00011.safetensors",
99
+ "model.layers.18.mlp.down_proj.weight": "model-00006-of-00011.safetensors",
100
+ "model.layers.18.mlp.gate_proj.weight": "model-00005-of-00011.safetensors",
101
+ "model.layers.18.mlp.up_proj.weight": "model-00006-of-00011.safetensors",
102
+ "model.layers.18.post_attention_layernorm.weight": "model-00006-of-00011.safetensors",
103
+ "model.layers.18.self_attn.k_proj.weight": "model-00005-of-00011.safetensors",
104
+ "model.layers.18.self_attn.o_proj.weight": "model-00005-of-00011.safetensors",
105
+ "model.layers.18.self_attn.q_proj.weight": "model-00005-of-00011.safetensors",
106
+ "model.layers.18.self_attn.v_proj.weight": "model-00005-of-00011.safetensors",
107
+ "model.layers.19.input_layernorm.weight": "model-00006-of-00011.safetensors",
108
+ "model.layers.19.mlp.down_proj.weight": "model-00006-of-00011.safetensors",
109
+ "model.layers.19.mlp.gate_proj.weight": "model-00006-of-00011.safetensors",
110
+ "model.layers.19.mlp.up_proj.weight": "model-00006-of-00011.safetensors",
111
+ "model.layers.19.post_attention_layernorm.weight": "model-00006-of-00011.safetensors",
112
+ "model.layers.19.self_attn.k_proj.weight": "model-00006-of-00011.safetensors",
113
+ "model.layers.19.self_attn.o_proj.weight": "model-00006-of-00011.safetensors",
114
+ "model.layers.19.self_attn.q_proj.weight": "model-00006-of-00011.safetensors",
115
+ "model.layers.19.self_attn.v_proj.weight": "model-00006-of-00011.safetensors",
116
+ "model.layers.2.input_layernorm.weight": "model-00001-of-00011.safetensors",
117
+ "model.layers.2.mlp.down_proj.weight": "model-00001-of-00011.safetensors",
118
+ "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00011.safetensors",
119
+ "model.layers.2.mlp.up_proj.weight": "model-00001-of-00011.safetensors",
120
+ "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00011.safetensors",
121
+ "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00011.safetensors",
122
+ "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00011.safetensors",
123
+ "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00011.safetensors",
124
+ "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00011.safetensors",
125
+ "model.layers.20.input_layernorm.weight": "model-00006-of-00011.safetensors",
126
+ "model.layers.20.mlp.down_proj.weight": "model-00006-of-00011.safetensors",
127
+ "model.layers.20.mlp.gate_proj.weight": "model-00006-of-00011.safetensors",
128
+ "model.layers.20.mlp.up_proj.weight": "model-00006-of-00011.safetensors",
129
+ "model.layers.20.post_attention_layernorm.weight": "model-00006-of-00011.safetensors",
130
+ "model.layers.20.self_attn.k_proj.weight": "model-00006-of-00011.safetensors",
131
+ "model.layers.20.self_attn.o_proj.weight": "model-00006-of-00011.safetensors",
132
+ "model.layers.20.self_attn.q_proj.weight": "model-00006-of-00011.safetensors",
133
+ "model.layers.20.self_attn.v_proj.weight": "model-00006-of-00011.safetensors",
134
+ "model.layers.21.input_layernorm.weight": "model-00006-of-00011.safetensors",
135
+ "model.layers.21.mlp.down_proj.weight": "model-00006-of-00011.safetensors",
136
+ "model.layers.21.mlp.gate_proj.weight": "model-00006-of-00011.safetensors",
137
+ "model.layers.21.mlp.up_proj.weight": "model-00006-of-00011.safetensors",
138
+ "model.layers.21.post_attention_layernorm.weight": "model-00006-of-00011.safetensors",
139
+ "model.layers.21.self_attn.k_proj.weight": "model-00006-of-00011.safetensors",
140
+ "model.layers.21.self_attn.o_proj.weight": "model-00006-of-00011.safetensors",
141
+ "model.layers.21.self_attn.q_proj.weight": "model-00006-of-00011.safetensors",
142
+ "model.layers.21.self_attn.v_proj.weight": "model-00006-of-00011.safetensors",
143
+ "model.layers.22.input_layernorm.weight": "model-00007-of-00011.safetensors",
144
+ "model.layers.22.mlp.down_proj.weight": "model-00007-of-00011.safetensors",
145
+ "model.layers.22.mlp.gate_proj.weight": "model-00007-of-00011.safetensors",
146
+ "model.layers.22.mlp.up_proj.weight": "model-00007-of-00011.safetensors",
147
+ "model.layers.22.post_attention_layernorm.weight": "model-00007-of-00011.safetensors",
148
+ "model.layers.22.self_attn.k_proj.weight": "model-00006-of-00011.safetensors",
149
+ "model.layers.22.self_attn.o_proj.weight": "model-00006-of-00011.safetensors",
150
+ "model.layers.22.self_attn.q_proj.weight": "model-00006-of-00011.safetensors",
151
+ "model.layers.22.self_attn.v_proj.weight": "model-00006-of-00011.safetensors",
152
+ "model.layers.23.input_layernorm.weight": "model-00007-of-00011.safetensors",
153
+ "model.layers.23.mlp.down_proj.weight": "model-00007-of-00011.safetensors",
154
+ "model.layers.23.mlp.gate_proj.weight": "model-00007-of-00011.safetensors",
155
+ "model.layers.23.mlp.up_proj.weight": "model-00007-of-00011.safetensors",
156
+ "model.layers.23.post_attention_layernorm.weight": "model-00007-of-00011.safetensors",
157
+ "model.layers.23.self_attn.k_proj.weight": "model-00007-of-00011.safetensors",
158
+ "model.layers.23.self_attn.o_proj.weight": "model-00007-of-00011.safetensors",
159
+ "model.layers.23.self_attn.q_proj.weight": "model-00007-of-00011.safetensors",
160
+ "model.layers.23.self_attn.v_proj.weight": "model-00007-of-00011.safetensors",
161
+ "model.layers.24.input_layernorm.weight": "model-00007-of-00011.safetensors",
162
+ "model.layers.24.mlp.down_proj.weight": "model-00007-of-00011.safetensors",
163
+ "model.layers.24.mlp.gate_proj.weight": "model-00007-of-00011.safetensors",
164
+ "model.layers.24.mlp.up_proj.weight": "model-00007-of-00011.safetensors",
165
+ "model.layers.24.post_attention_layernorm.weight": "model-00007-of-00011.safetensors",
166
+ "model.layers.24.self_attn.k_proj.weight": "model-00007-of-00011.safetensors",
167
+ "model.layers.24.self_attn.o_proj.weight": "model-00007-of-00011.safetensors",
168
+ "model.layers.24.self_attn.q_proj.weight": "model-00007-of-00011.safetensors",
169
+ "model.layers.24.self_attn.v_proj.weight": "model-00007-of-00011.safetensors",
170
+ "model.layers.25.input_layernorm.weight": "model-00007-of-00011.safetensors",
171
+ "model.layers.25.mlp.down_proj.weight": "model-00007-of-00011.safetensors",
172
+ "model.layers.25.mlp.gate_proj.weight": "model-00007-of-00011.safetensors",
173
+ "model.layers.25.mlp.up_proj.weight": "model-00007-of-00011.safetensors",
174
+ "model.layers.25.post_attention_layernorm.weight": "model-00007-of-00011.safetensors",
175
+ "model.layers.25.self_attn.k_proj.weight": "model-00007-of-00011.safetensors",
176
+ "model.layers.25.self_attn.o_proj.weight": "model-00007-of-00011.safetensors",
177
+ "model.layers.25.self_attn.q_proj.weight": "model-00007-of-00011.safetensors",
178
+ "model.layers.25.self_attn.v_proj.weight": "model-00007-of-00011.safetensors",
179
+ "model.layers.26.input_layernorm.weight": "model-00008-of-00011.safetensors",
180
+ "model.layers.26.mlp.down_proj.weight": "model-00008-of-00011.safetensors",
181
+ "model.layers.26.mlp.gate_proj.weight": "model-00008-of-00011.safetensors",
182
+ "model.layers.26.mlp.up_proj.weight": "model-00008-of-00011.safetensors",
183
+ "model.layers.26.post_attention_layernorm.weight": "model-00008-of-00011.safetensors",
184
+ "model.layers.26.self_attn.k_proj.weight": "model-00007-of-00011.safetensors",
185
+ "model.layers.26.self_attn.o_proj.weight": "model-00008-of-00011.safetensors",
186
+ "model.layers.26.self_attn.q_proj.weight": "model-00007-of-00011.safetensors",
187
+ "model.layers.26.self_attn.v_proj.weight": "model-00007-of-00011.safetensors",
188
+ "model.layers.27.input_layernorm.weight": "model-00008-of-00011.safetensors",
189
+ "model.layers.27.mlp.down_proj.weight": "model-00008-of-00011.safetensors",
190
+ "model.layers.27.mlp.gate_proj.weight": "model-00008-of-00011.safetensors",
191
+ "model.layers.27.mlp.up_proj.weight": "model-00008-of-00011.safetensors",
192
+ "model.layers.27.post_attention_layernorm.weight": "model-00008-of-00011.safetensors",
193
+ "model.layers.27.self_attn.k_proj.weight": "model-00008-of-00011.safetensors",
194
+ "model.layers.27.self_attn.o_proj.weight": "model-00008-of-00011.safetensors",
195
+ "model.layers.27.self_attn.q_proj.weight": "model-00008-of-00011.safetensors",
196
+ "model.layers.27.self_attn.v_proj.weight": "model-00008-of-00011.safetensors",
197
+ "model.layers.28.input_layernorm.weight": "model-00008-of-00011.safetensors",
198
+ "model.layers.28.mlp.down_proj.weight": "model-00008-of-00011.safetensors",
199
+ "model.layers.28.mlp.gate_proj.weight": "model-00008-of-00011.safetensors",
200
+ "model.layers.28.mlp.up_proj.weight": "model-00008-of-00011.safetensors",
201
+ "model.layers.28.post_attention_layernorm.weight": "model-00008-of-00011.safetensors",
202
+ "model.layers.28.self_attn.k_proj.weight": "model-00008-of-00011.safetensors",
203
+ "model.layers.28.self_attn.o_proj.weight": "model-00008-of-00011.safetensors",
204
+ "model.layers.28.self_attn.q_proj.weight": "model-00008-of-00011.safetensors",
205
+ "model.layers.28.self_attn.v_proj.weight": "model-00008-of-00011.safetensors",
206
+ "model.layers.29.input_layernorm.weight": "model-00008-of-00011.safetensors",
207
+ "model.layers.29.mlp.down_proj.weight": "model-00008-of-00011.safetensors",
208
+ "model.layers.29.mlp.gate_proj.weight": "model-00008-of-00011.safetensors",
209
+ "model.layers.29.mlp.up_proj.weight": "model-00008-of-00011.safetensors",
210
+ "model.layers.29.post_attention_layernorm.weight": "model-00008-of-00011.safetensors",
211
+ "model.layers.29.self_attn.k_proj.weight": "model-00008-of-00011.safetensors",
212
+ "model.layers.29.self_attn.o_proj.weight": "model-00008-of-00011.safetensors",
213
+ "model.layers.29.self_attn.q_proj.weight": "model-00008-of-00011.safetensors",
214
+ "model.layers.29.self_attn.v_proj.weight": "model-00008-of-00011.safetensors",
215
+ "model.layers.3.input_layernorm.weight": "model-00002-of-00011.safetensors",
216
+ "model.layers.3.mlp.down_proj.weight": "model-00002-of-00011.safetensors",
217
+ "model.layers.3.mlp.gate_proj.weight": "model-00002-of-00011.safetensors",
218
+ "model.layers.3.mlp.up_proj.weight": "model-00002-of-00011.safetensors",
219
+ "model.layers.3.post_attention_layernorm.weight": "model-00002-of-00011.safetensors",
220
+ "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00011.safetensors",
221
+ "model.layers.3.self_attn.o_proj.weight": "model-00002-of-00011.safetensors",
222
+ "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00011.safetensors",
223
+ "model.layers.3.self_attn.v_proj.weight": "model-00002-of-00011.safetensors",
224
+ "model.layers.30.input_layernorm.weight": "model-00009-of-00011.safetensors",
225
+ "model.layers.30.mlp.down_proj.weight": "model-00009-of-00011.safetensors",
226
+ "model.layers.30.mlp.gate_proj.weight": "model-00009-of-00011.safetensors",
227
+ "model.layers.30.mlp.up_proj.weight": "model-00009-of-00011.safetensors",
228
+ "model.layers.30.post_attention_layernorm.weight": "model-00009-of-00011.safetensors",
229
+ "model.layers.30.self_attn.k_proj.weight": "model-00008-of-00011.safetensors",
230
+ "model.layers.30.self_attn.o_proj.weight": "model-00009-of-00011.safetensors",
231
+ "model.layers.30.self_attn.q_proj.weight": "model-00008-of-00011.safetensors",
232
+ "model.layers.30.self_attn.v_proj.weight": "model-00009-of-00011.safetensors",
233
+ "model.layers.31.input_layernorm.weight": "model-00009-of-00011.safetensors",
234
+ "model.layers.31.mlp.down_proj.weight": "model-00009-of-00011.safetensors",
235
+ "model.layers.31.mlp.gate_proj.weight": "model-00009-of-00011.safetensors",
236
+ "model.layers.31.mlp.up_proj.weight": "model-00009-of-00011.safetensors",
237
+ "model.layers.31.post_attention_layernorm.weight": "model-00009-of-00011.safetensors",
238
+ "model.layers.31.self_attn.k_proj.weight": "model-00009-of-00011.safetensors",
239
+ "model.layers.31.self_attn.o_proj.weight": "model-00009-of-00011.safetensors",
240
+ "model.layers.31.self_attn.q_proj.weight": "model-00009-of-00011.safetensors",
241
+ "model.layers.31.self_attn.v_proj.weight": "model-00009-of-00011.safetensors",
242
+ "model.layers.32.input_layernorm.weight": "model-00009-of-00011.safetensors",
243
+ "model.layers.32.mlp.down_proj.weight": "model-00009-of-00011.safetensors",
244
+ "model.layers.32.mlp.gate_proj.weight": "model-00009-of-00011.safetensors",
245
+ "model.layers.32.mlp.up_proj.weight": "model-00009-of-00011.safetensors",
246
+ "model.layers.32.post_attention_layernorm.weight": "model-00009-of-00011.safetensors",
247
+ "model.layers.32.self_attn.k_proj.weight": "model-00009-of-00011.safetensors",
248
+ "model.layers.32.self_attn.o_proj.weight": "model-00009-of-00011.safetensors",
249
+ "model.layers.32.self_attn.q_proj.weight": "model-00009-of-00011.safetensors",
250
+ "model.layers.32.self_attn.v_proj.weight": "model-00009-of-00011.safetensors",
251
+ "model.layers.33.input_layernorm.weight": "model-00009-of-00011.safetensors",
252
+ "model.layers.33.mlp.down_proj.weight": "model-00009-of-00011.safetensors",
253
+ "model.layers.33.mlp.gate_proj.weight": "model-00009-of-00011.safetensors",
254
+ "model.layers.33.mlp.up_proj.weight": "model-00009-of-00011.safetensors",
255
+ "model.layers.33.post_attention_layernorm.weight": "model-00009-of-00011.safetensors",
256
+ "model.layers.33.self_attn.k_proj.weight": "model-00009-of-00011.safetensors",
257
+ "model.layers.33.self_attn.o_proj.weight": "model-00009-of-00011.safetensors",
258
+ "model.layers.33.self_attn.q_proj.weight": "model-00009-of-00011.safetensors",
259
+ "model.layers.33.self_attn.v_proj.weight": "model-00009-of-00011.safetensors",
260
+ "model.layers.34.input_layernorm.weight": "model-00010-of-00011.safetensors",
261
+ "model.layers.34.mlp.down_proj.weight": "model-00010-of-00011.safetensors",
262
+ "model.layers.34.mlp.gate_proj.weight": "model-00010-of-00011.safetensors",
263
+ "model.layers.34.mlp.up_proj.weight": "model-00010-of-00011.safetensors",
264
+ "model.layers.34.post_attention_layernorm.weight": "model-00010-of-00011.safetensors",
265
+ "model.layers.34.self_attn.k_proj.weight": "model-00010-of-00011.safetensors",
266
+ "model.layers.34.self_attn.o_proj.weight": "model-00010-of-00011.safetensors",
267
+ "model.layers.34.self_attn.q_proj.weight": "model-00009-of-00011.safetensors",
268
+ "model.layers.34.self_attn.v_proj.weight": "model-00010-of-00011.safetensors",
269
+ "model.layers.35.input_layernorm.weight": "model-00010-of-00011.safetensors",
270
+ "model.layers.35.mlp.down_proj.weight": "model-00010-of-00011.safetensors",
271
+ "model.layers.35.mlp.gate_proj.weight": "model-00010-of-00011.safetensors",
272
+ "model.layers.35.mlp.up_proj.weight": "model-00010-of-00011.safetensors",
273
+ "model.layers.35.post_attention_layernorm.weight": "model-00010-of-00011.safetensors",
274
+ "model.layers.35.self_attn.k_proj.weight": "model-00010-of-00011.safetensors",
275
+ "model.layers.35.self_attn.o_proj.weight": "model-00010-of-00011.safetensors",
276
+ "model.layers.35.self_attn.q_proj.weight": "model-00010-of-00011.safetensors",
277
+ "model.layers.35.self_attn.v_proj.weight": "model-00010-of-00011.safetensors",
278
+ "model.layers.36.input_layernorm.weight": "model-00010-of-00011.safetensors",
279
+ "model.layers.36.mlp.down_proj.weight": "model-00010-of-00011.safetensors",
280
+ "model.layers.36.mlp.gate_proj.weight": "model-00010-of-00011.safetensors",
281
+ "model.layers.36.mlp.up_proj.weight": "model-00010-of-00011.safetensors",
282
+ "model.layers.36.post_attention_layernorm.weight": "model-00010-of-00011.safetensors",
283
+ "model.layers.36.self_attn.k_proj.weight": "model-00010-of-00011.safetensors",
284
+ "model.layers.36.self_attn.o_proj.weight": "model-00010-of-00011.safetensors",
285
+ "model.layers.36.self_attn.q_proj.weight": "model-00010-of-00011.safetensors",
286
+ "model.layers.36.self_attn.v_proj.weight": "model-00010-of-00011.safetensors",
287
+ "model.layers.37.input_layernorm.weight": "model-00010-of-00011.safetensors",
288
+ "model.layers.37.mlp.down_proj.weight": "model-00010-of-00011.safetensors",
289
+ "model.layers.37.mlp.gate_proj.weight": "model-00010-of-00011.safetensors",
290
+ "model.layers.37.mlp.up_proj.weight": "model-00010-of-00011.safetensors",
291
+ "model.layers.37.post_attention_layernorm.weight": "model-00010-of-00011.safetensors",
292
+ "model.layers.37.self_attn.k_proj.weight": "model-00010-of-00011.safetensors",
293
+ "model.layers.37.self_attn.o_proj.weight": "model-00010-of-00011.safetensors",
294
+ "model.layers.37.self_attn.q_proj.weight": "model-00010-of-00011.safetensors",
295
+ "model.layers.37.self_attn.v_proj.weight": "model-00010-of-00011.safetensors",
296
+ "model.layers.38.input_layernorm.weight": "model-00011-of-00011.safetensors",
297
+ "model.layers.38.mlp.down_proj.weight": "model-00011-of-00011.safetensors",
298
+ "model.layers.38.mlp.gate_proj.weight": "model-00011-of-00011.safetensors",
299
+ "model.layers.38.mlp.up_proj.weight": "model-00011-of-00011.safetensors",
300
+ "model.layers.38.post_attention_layernorm.weight": "model-00011-of-00011.safetensors",
301
+ "model.layers.38.self_attn.k_proj.weight": "model-00011-of-00011.safetensors",
302
+ "model.layers.38.self_attn.o_proj.weight": "model-00011-of-00011.safetensors",
303
+ "model.layers.38.self_attn.q_proj.weight": "model-00011-of-00011.safetensors",
304
+ "model.layers.38.self_attn.v_proj.weight": "model-00011-of-00011.safetensors",
305
+ "model.layers.39.input_layernorm.weight": "model-00011-of-00011.safetensors",
306
+ "model.layers.39.mlp.down_proj.weight": "model-00011-of-00011.safetensors",
307
+ "model.layers.39.mlp.gate_proj.weight": "model-00011-of-00011.safetensors",
308
+ "model.layers.39.mlp.up_proj.weight": "model-00011-of-00011.safetensors",
309
+ "model.layers.39.post_attention_layernorm.weight": "model-00011-of-00011.safetensors",
310
+ "model.layers.39.self_attn.k_proj.weight": "model-00011-of-00011.safetensors",
311
+ "model.layers.39.self_attn.o_proj.weight": "model-00011-of-00011.safetensors",
312
+ "model.layers.39.self_attn.q_proj.weight": "model-00011-of-00011.safetensors",
313
+ "model.layers.39.self_attn.v_proj.weight": "model-00011-of-00011.safetensors",
314
+ "model.layers.4.input_layernorm.weight": "model-00002-of-00011.safetensors",
315
+ "model.layers.4.mlp.down_proj.weight": "model-00002-of-00011.safetensors",
316
+ "model.layers.4.mlp.gate_proj.weight": "model-00002-of-00011.safetensors",
317
+ "model.layers.4.mlp.up_proj.weight": "model-00002-of-00011.safetensors",
318
+ "model.layers.4.post_attention_layernorm.weight": "model-00002-of-00011.safetensors",
319
+ "model.layers.4.self_attn.k_proj.weight": "model-00002-of-00011.safetensors",
320
+ "model.layers.4.self_attn.o_proj.weight": "model-00002-of-00011.safetensors",
321
+ "model.layers.4.self_attn.q_proj.weight": "model-00002-of-00011.safetensors",
322
+ "model.layers.4.self_attn.v_proj.weight": "model-00002-of-00011.safetensors",
323
+ "model.layers.5.input_layernorm.weight": "model-00002-of-00011.safetensors",
324
+ "model.layers.5.mlp.down_proj.weight": "model-00002-of-00011.safetensors",
325
+ "model.layers.5.mlp.gate_proj.weight": "model-00002-of-00011.safetensors",
326
+ "model.layers.5.mlp.up_proj.weight": "model-00002-of-00011.safetensors",
327
+ "model.layers.5.post_attention_layernorm.weight": "model-00002-of-00011.safetensors",
328
+ "model.layers.5.self_attn.k_proj.weight": "model-00002-of-00011.safetensors",
329
+ "model.layers.5.self_attn.o_proj.weight": "model-00002-of-00011.safetensors",
330
+ "model.layers.5.self_attn.q_proj.weight": "model-00002-of-00011.safetensors",
331
+ "model.layers.5.self_attn.v_proj.weight": "model-00002-of-00011.safetensors",
332
+ "model.layers.6.input_layernorm.weight": "model-00002-of-00011.safetensors",
333
+ "model.layers.6.mlp.down_proj.weight": "model-00002-of-00011.safetensors",
334
+ "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00011.safetensors",
335
+ "model.layers.6.mlp.up_proj.weight": "model-00002-of-00011.safetensors",
336
+ "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00011.safetensors",
337
+ "model.layers.6.self_attn.k_proj.weight": "model-00002-of-00011.safetensors",
338
+ "model.layers.6.self_attn.o_proj.weight": "model-00002-of-00011.safetensors",
339
+ "model.layers.6.self_attn.q_proj.weight": "model-00002-of-00011.safetensors",
340
+ "model.layers.6.self_attn.v_proj.weight": "model-00002-of-00011.safetensors",
341
+ "model.layers.7.input_layernorm.weight": "model-00003-of-00011.safetensors",
342
+ "model.layers.7.mlp.down_proj.weight": "model-00003-of-00011.safetensors",
343
+ "model.layers.7.mlp.gate_proj.weight": "model-00003-of-00011.safetensors",
344
+ "model.layers.7.mlp.up_proj.weight": "model-00003-of-00011.safetensors",
345
+ "model.layers.7.post_attention_layernorm.weight": "model-00003-of-00011.safetensors",
346
+ "model.layers.7.self_attn.k_proj.weight": "model-00003-of-00011.safetensors",
347
+ "model.layers.7.self_attn.o_proj.weight": "model-00003-of-00011.safetensors",
348
+ "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00011.safetensors",
349
+ "model.layers.7.self_attn.v_proj.weight": "model-00003-of-00011.safetensors",
350
+ "model.layers.8.input_layernorm.weight": "model-00003-of-00011.safetensors",
351
+ "model.layers.8.mlp.down_proj.weight": "model-00003-of-00011.safetensors",
352
+ "model.layers.8.mlp.gate_proj.weight": "model-00003-of-00011.safetensors",
353
+ "model.layers.8.mlp.up_proj.weight": "model-00003-of-00011.safetensors",
354
+ "model.layers.8.post_attention_layernorm.weight": "model-00003-of-00011.safetensors",
355
+ "model.layers.8.self_attn.k_proj.weight": "model-00003-of-00011.safetensors",
356
+ "model.layers.8.self_attn.o_proj.weight": "model-00003-of-00011.safetensors",
357
+ "model.layers.8.self_attn.q_proj.weight": "model-00003-of-00011.safetensors",
358
+ "model.layers.8.self_attn.v_proj.weight": "model-00003-of-00011.safetensors",
359
+ "model.layers.9.input_layernorm.weight": "model-00003-of-00011.safetensors",
360
+ "model.layers.9.mlp.down_proj.weight": "model-00003-of-00011.safetensors",
361
+ "model.layers.9.mlp.gate_proj.weight": "model-00003-of-00011.safetensors",
362
+ "model.layers.9.mlp.up_proj.weight": "model-00003-of-00011.safetensors",
363
+ "model.layers.9.post_attention_layernorm.weight": "model-00003-of-00011.safetensors",
364
+ "model.layers.9.self_attn.k_proj.weight": "model-00003-of-00011.safetensors",
365
+ "model.layers.9.self_attn.o_proj.weight": "model-00003-of-00011.safetensors",
366
+ "model.layers.9.self_attn.q_proj.weight": "model-00003-of-00011.safetensors",
367
+ "model.layers.9.self_attn.v_proj.weight": "model-00003-of-00011.safetensors",
368
+ "model.norm.weight": "model-00011-of-00011.safetensors"
369
+ }
370
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token": {
3
+ "content": "<s>",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "eos_token": {
10
+ "content": "</s>",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "unk_token": {
17
+ "content": "<unk>",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ }
23
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0a4a2876dcbbbeaa71fd99d9cf32fe5f353b4e8976acbd44f0d0f674ba2275b
3
+ size 760271
tokenizer_config.json ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "add_prefix_space": true,
5
+ "added_tokens_decoder": {
6
+ "0": {
7
+ "content": "<unk>",
8
+ "lstrip": false,
9
+ "normalized": false,
10
+ "rstrip": false,
11
+ "single_word": false,
12
+ "special": true
13
+ },
14
+ "1": {
15
+ "content": "<s>",
16
+ "lstrip": false,
17
+ "normalized": false,
18
+ "rstrip": false,
19
+ "single_word": false,
20
+ "special": true
21
+ },
22
+ "2": {
23
+ "content": "</s>",
24
+ "lstrip": false,
25
+ "normalized": false,
26
+ "rstrip": false,
27
+ "single_word": false,
28
+ "special": true
29
+ }
30
+ },
31
+ "bos_token": "<s>",
32
+ "clean_up_tokenization_spaces": false,
33
+ "eos_token": "</s>",
34
+ "legacy": true,
35
+ "model_max_length": 1000000000000000019884624838656,
36
+ "pad_token": null,
37
+ "sp_model_kwargs": {},
38
+ "spaces_between_special_tokens": false,
39
+ "tokenizer_class": "LlamaTokenizer",
40
+ "unk_token": "<unk>",
41
+ "use_default_system_prompt": false
42
+ }