Hugging Face's logo

FiditeNemini
/
Llama3-FiditeNemini-120B

Text Generation
Transformers
Safetensors
llama
mergekit
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
FiditeNemini commited on
Commit
dd6f8c3
•
1 Parent(s): 9f8838d

Upload folder using huggingface_hub

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. .DS_Store +0 -0
  2. README.md +54 -0
  3. config.json +28 -0
  4. model-00001-of-00050.safetensors +3 -0
  5. model-00002-of-00050.safetensors +3 -0
  6. model-00003-of-00050.safetensors +3 -0
  7. model-00004-of-00050.safetensors +3 -0
  8. model-00005-of-00050.safetensors +3 -0
  9. model-00006-of-00050.safetensors +3 -0
  10. model-00007-of-00050.safetensors +3 -0
  11. model-00008-of-00050.safetensors +3 -0
  12. model-00009-of-00050.safetensors +3 -0
  13. model-00010-of-00050.safetensors +3 -0
  14. model-00011-of-00050.safetensors +3 -0
  15. model-00012-of-00050.safetensors +3 -0
  16. model-00013-of-00050.safetensors +3 -0
  17. model-00014-of-00050.safetensors +3 -0
  18. model-00015-of-00050.safetensors +3 -0
  19. model-00016-of-00050.safetensors +3 -0
  20. model-00017-of-00050.safetensors +3 -0
  21. model-00018-of-00050.safetensors +3 -0
  22. model-00019-of-00050.safetensors +3 -0
  23. model-00020-of-00050.safetensors +3 -0
  24. model-00021-of-00050.safetensors +3 -0
  25. model-00022-of-00050.safetensors +3 -0
  26. model-00023-of-00050.safetensors +3 -0
  27. model-00024-of-00050.safetensors +3 -0
  28. model-00025-of-00050.safetensors +3 -0
  29. model-00026-of-00050.safetensors +3 -0
  30. model-00027-of-00050.safetensors +3 -0
  31. model-00028-of-00050.safetensors +3 -0
  32. model-00029-of-00050.safetensors +3 -0
  33. model-00030-of-00050.safetensors +3 -0
  34. model-00031-of-00050.safetensors +3 -0
  35. model-00032-of-00050.safetensors +3 -0
  36. model-00033-of-00050.safetensors +3 -0
  37. model-00034-of-00050.safetensors +3 -0
  38. model-00035-of-00050.safetensors +3 -0
  39. model-00036-of-00050.safetensors +3 -0
  40. model-00037-of-00050.safetensors +3 -0
  41. model-00038-of-00050.safetensors +3 -0
  42. model-00039-of-00050.safetensors +3 -0
  43. model-00040-of-00050.safetensors +3 -0
  44. model-00041-of-00050.safetensors +3 -0
  45. model-00042-of-00050.safetensors +3 -0
  46. model-00043-of-00050.safetensors +3 -0
  47. model-00044-of-00050.safetensors +3 -0
  48. model-00045-of-00050.safetensors +3 -0
  49. model-00046-of-00050.safetensors +3 -0
  50. model-00047-of-00050.safetensors +3 -0
.DS_Store ADDED
Binary file (6.15 kB). View file
 
README.md ADDED
@@ -0,0 +1,54 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: []
3
+ library_name: transformers
4
+ tags:
5
+ - mergekit
6
+ - merge
7
+
8
+ ---
9
+ ![llama3 fiditenemini logo](llama3-fiditenemini-logo.png)
10
+
11
+ # Unhinged-Llama3-120B
12
+
13
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
14
+
15
+ ## Merge Details
16
+ ### Merge Method
17
+
18
+ This model was merged using the passthrough merge method.
19
+
20
+ ### Models Merged
21
+
22
+ The following models were included in the merge:
23
+ * FiditeNemini/Llama3-FiditeNemini-70B-Source
24
+
25
+ ### Configuration
26
+
27
+ The following YAML configuration was used to produce this model:
28
+
29
+ ```yaml
30
+ dtype: bfloat16
31
+ merge_method: passthrough
32
+ slices:
33
+ - sources:
34
+ - layer_range: [0, 20]
35
+ model: FiditeNemini/Llama3-FiditeNemini-70B-Source
36
+ - sources:
37
+ - layer_range: [10, 30]
38
+ model: FiditeNemini/Llama3-FiditeNemini-70B-Source
39
+ - sources:
40
+ - layer_range: [20, 40]
41
+ model: FiditeNemini/Llama3-FiditeNemini-70B-Source
42
+ - sources:
43
+ - layer_range: [30, 50]
44
+ model: FiditeNemini/Llama3-FiditeNemini-70B-Source
45
+ - sources:
46
+ - layer_range: [40, 60]
47
+ model: FiditeNemini/Llama3-FiditeNemini-70B-Source
48
+ - sources:
49
+ - layer_range: [50, 70]
50
+ model: FiditeNemini/Llama3-FiditeNemini-70B-Source
51
+ - sources:
52
+ - layer_range: [60, 80]
53
+ model: FiditeNemini/Llama3-FiditeNemini-70B-Source
54
+ ```
config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/Users/willdee/Documents/Projects/llama.cpp/models/Llama3-FiditeNemini-70B",
3
+ "architectures": [
4
+ "LlamaForCausalLM"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 128000,
9
+ "eos_token_id": 128001,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 8192,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 28672,
14
+ "max_position_embeddings": 8192,
15
+ "model_type": "llama",
16
+ "num_attention_heads": 64,
17
+ "num_hidden_layers": 140,
18
+ "num_key_value_heads": 8,
19
+ "pretraining_tp": 1,
20
+ "rms_norm_eps": 1e-05,
21
+ "rope_scaling": null,
22
+ "rope_theta": 500000.0,
23
+ "tie_word_embeddings": false,
24
+ "torch_dtype": "bfloat16",
25
+ "transformers_version": "4.40.2",
26
+ "use_cache": true,
27
+ "vocab_size": 128256
28
+ }
model-00001-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e6d78fe2cc50fe5afa2a38330d006ed89a523721607bf95c33faebe2d14c829
3
+ size 4672471504
model-00002-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5ad6435e7bcafcbe0bb3c38afaf7f0602cd5113ea6eed9ac27cbe0613411eb76
3
+ size 4831922704
model-00003-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0b519f788bc1f732526b801f6af64c4b551349d9cc67b159ad2f8d38a3d6ab59
3
+ size 4966190768
model-00004-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6666de55971628988899a6ba13af35cb7b6ba3c754e55a26e3eb1c4f84dcba64
3
+ size 4831906224
model-00005-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6efd75d0c373ac0318572ef301a58dd29a08c5f0dcd38569f1783f95c9f4c733
3
+ size 4999711744
model-00006-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:84d8bb7177453366dc487f5bc899f34005439cda3598ceda0a3f0cf62f0b3b6c
3
+ size 4798385256
model-00007-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d15cfc94d06e9410d475508af1795ea6eaed398865c27d9d1f27f2585d7efb9
3
+ size 4831906216
model-00008-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8014b2ee45e9a944d16c8e0b74b57a695106c78251c6f78bd4027389dc47925e
3
+ size 4966190768
model-00009-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:346a26ff27dfb2201cd22db4a974ddf4dfd92431479240b82bc17bbf1d7a07d3
3
+ size 4664167408
model-00010-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6dc11f541da031ccf07624b1e7ffbce2a39685b40c336c942f2a5790c9abea8c
3
+ size 4831906224
model-00011-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a34d4ba854c47a2414fb491390a94643fc1f92a05c13360fd46f5aca9dec4215
3
+ size 4999711744
model-00012-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:645e285e9b07785fafde1f8bd8baab1ff942b2919bf85144fe07cb53645d1aa6
3
+ size 4798385256
model-00013-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:be9978c376ead9550f565c988da64e4905cd6f386bf0b34794875ff605f35b32
3
+ size 4831906216
model-00014-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:72315f15ec1c315c812f20e4d692f512923e93c0ab817e81e8dbf2f0a45c1e41
3
+ size 4966190768
model-00015-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cd5c99842756a04d78cfcb041a1b82e13a6cca239078c76371ad5b06fe9d8a0c
3
+ size 4831906224
model-00016-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8ff4b7ea7171d0b901781b316c71474892ba067e28e93baee16eec38567bb731
3
+ size 4966174264
model-00017-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5d184e3ebce497c3b9cf67348586ce49dada5d4a6893c1287df440875e88bc91
3
+ size 4966140568
model-00018-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:638b926a14bc5a6ce11f7e47102662845b832c5477d39d8aafd3ae736e6f0fd3
3
+ size 4999711728
model-00019-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97235ff44fe2aa2663dc794047f48b22c7529c42f18d8dae7453e34d11878a6b
3
+ size 4999711744
model-00020-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:18c6954bde01ea772d2b4201cd17e43845c955a0a032b827fcfee7501f531fb9
3
+ size 4798385256
model-00021-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e11a0e78954c57b1d5d66993eb7ba148df540fd1cc75855bcc485ffc497ad587
3
+ size 4831906216
model-00022-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7cef067220875f695af9bc7690bfc8995d5c738a695d94446d8126dae2ac4bb6
3
+ size 4966190768
model-00023-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c39df073e1abce05d61b8de28af656827c9aec306e590bfd6150af5a0cd4272
3
+ size 4831906224
model-00024-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a71c11fa164b54373e0b111ae5a174a63238599d4b1cf2c22e769cd0809eb5bd
3
+ size 4664167400
model-00025-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:85db595226d9b1d69cf88c6a6703fb0919ee18b3bbbd56fd2b84cacca5e9dbd6
3
+ size 4999711744
model-00026-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2e86ab65877b7693de745326ceeea01583d0357b60c49da01174681ec528353b
3
+ size 4798385256
model-00027-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8926fd2ea7ebf910d79120ba215f4a1252064ed85d5eef2b8415991a2df8bddb
3
+ size 4831906216
model-00028-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c320a65c1e716f5e3062070b15b1aeb8795641db19a9f34790afe65d2ee0457c
3
+ size 4966190768
model-00029-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5269b7326c4dd163851b31f234fc7fd439c66b1961ff4c5d5366f05b0d60f3f3
3
+ size 4831906224
model-00030-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc6555215209bf0ae0709703abbee7b8c4b5ed2c6b22fb22dffbb6793376dd40
3
+ size 4966190768
model-00031-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b660fcfe34c043d6bde8f3d74be84c543f044f4ecb558f4066fcecc00b7cafc8
3
+ size 4982901392
model-00032-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e9dbd72bb8f762877e75fc59ea5d04df6c5a091c7936f10d9b9302a383585094
3
+ size 4982934400
model-00033-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d8279fd9072b881d324fa2f950b93427f3a94c08c154a87be3bb23187063169
3
+ size 4999711744
model-00034-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:df2032f3c33b91701603347ac24c37a266a449bf89d3fb8f12da27a94d774015
3
+ size 4798385264
model-00035-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1709598f31d62c82bf7af3385e3d4ba1b642533fe52d15a0cec3d0eca3945197
3
+ size 4831906232
model-00036-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4c0e4f9c0cee049a5822996bd043a6f1b0af864e71a7c9a13064bafa7ae4ee58
3
+ size 4966190776
model-00037-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f0d51f48067571a42d3108e27f4a034240c850a4b63ed58be252e993bbc1c45c
3
+ size 4831906232
model-00038-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c27d22ab2642f3d2dcdd30fe5aec35efdf8af03c78a01ac5106fbac5c699ce66
3
+ size 4966174288
model-00039-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:626cb520cabbe84faf31ae3c22a1142cca26030758836befe1a4c43d881837ea
3
+ size 4966140568
model-00040-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b95d95e895f12dc4fbe7283029910033c874db6000ab1007b0e575d73944008
3
+ size 4999711752
model-00041-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:213ba181c2ef3d0378093449cd19374c3a55a95ba420d9c5e87eaf92f98403b5
3
+ size 4999711768
model-00042-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:694dc50e393174b404dfe7dd1d6cdc30d6ba22ee77bb89d268a565a1bc71782f
3
+ size 4798385280
model-00043-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8d2ade5765ddd95bb400fd704c70faab3e478ef5b9e350f941988363bd275b2a
3
+ size 4831906240
model-00044-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:378e313d6cf9d5d44963274d082f98a8819609f13ac24ceede318bf3007ca238
3
+ size 4966190800
model-00045-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c9cccd4857369059b4f108e9269c6e6870d1d6ac0b3e57b3f96f781ee36388db
3
+ size 4831906248
model-00046-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3daf0c89ab8eb09b5a0775798bd03f17579c9d3e60bed810d2c2246ccf666bf
3
+ size 4966174288
model-00047-of-00050.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:49d21b7e9d25ffffba7d7a0e9cb9604999d3854ed32d132e7b119f3ee5c11dfb
3
+ size 4982917928