Upload MistralForCausalLM

README.md

@@ -0,0 +1,199 @@
+---
+library_name: transformers
+tags: []
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]

config.json

@@ -0,0 +1,27 @@
+{
+  "_name_or_path": "mistralai/Mistral-Large-Instruct-2411",
+  "architectures": [
+    "MistralForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 12288,
+  "initializer_range": 0.02,
+  "intermediate_size": 28672,
+  "max_position_embeddings": 131072,
+  "model_type": "mistral",
+  "num_attention_heads": 96,
+  "num_hidden_layers": 88,
+  "num_key_value_heads": 8,
+  "rms_norm_eps": 1e-05,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "torch_dtype": "bfloat16",
+  "transformers_version": "4.48.0",
+  "use_cache": true,
+  "vocab_size": 32768
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "transformers_version": "4.48.0"
+}

model-00001-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4a39f759922c8646441734e083f0aee1e81e82f3adba640ab005e9f15857d6b
+size 4932552432

model-00002-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8272b6b2f60b1b4c00c604f1776677190e64e8c637031d6d5e6aef539d481fa
+size 4831938528

model-00003-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95955010231ba22902041de7cd0e103bd3ce3096a59ea38ed4c2f4538bb6c18d
+size 4882269816

model-00004-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30bb55b1f8bbfa60ea0d657f650f259d25032a3bf04f1c15dab6e29650bb3fdd
+size 4831889136

model-00005-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37e2847d453b490d57e693c8153679f003c22f7a28eb5a50363bc2d2e85e66a5
+size 4831938520

model-00006-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c2697893f74d7b6cb8c9dc659c123d2461c18f515676b07351089d34d3f4823
+size 4831938528

model-00007-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35125e7705ac136ffc820a8c0f395ffaaaa0237d31a83cc33433f2c3065c42f4
+size 4882269832

model-00008-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e2b2b8e4bcb3b513f39e83749d48f9a41029e61e0f68c3858d7cd4443e64c4b
+size 4831889152

model-00009-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c208cc41915fbdd2600856f177894112fef2315a87fdbcd96ebc423b647ffe27
+size 4831938536

model-00010-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33a1c08ef69605bb9604c0f3b9aac17c83cd3ad6574a76c90ade2a76642799c0
+size 4831938544

model-00011-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6c6176e952af518248de37d0b28ee64e9274fcd72729fd5fcff462dfb531b2c
+size 4882269832

model-00012-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9d3dc2c52ee77f34b16e81b7a2def79d2cc0abe8241b6f55e4f89eab6e89084
+size 4831889152

model-00013-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:310f8c3e49566dfde7cba59b8502b56a68f60b4e0bd6b7c3d282c4b34cca8813
+size 4831938536

model-00014-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cca478f74407c71b2c812fe5cac0849921e2ad501942fb8106902adc74ea9cde
+size 4831938544

model-00015-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03836896f6c6956574c18ecdf2a5076240996ee8a677bb448049158fd4b2f22a
+size 4882269832

model-00016-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbc402b2548246e19957818605f10ecd19ce5895529126591f512893a1486104
+size 4831889152

model-00017-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c17d04ea39ef6b7284b191849c217932e698e45177fc3edd3703928a64b6953d
+size 4831938536

model-00018-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31337fa51a02f70c6ecf8fc5dc34c900f445c2de0feaa7bbde87ad4c1f795360
+size 4831938544

model-00019-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2eaa4661deb9ae7199abcaa1871863eb4da07d1895611516ce311c384ea76fe
+size 4882269832

model-00020-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e1b6a9b2634c6d2a7232ca006bf8a69fba9e886d2cf7e3314dc06ab4bb2d07f9
+size 4831889152

model-00021-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7580b1ba8bc044ddee879084a35304af72613d4f32905d8789cc202629b42d0b
+size 4831938536

model-00022-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:46887b62a3b2b14e4b81a6d523f0138b9b548bf825a48baa76c07ad9301d090d
+size 4831938544

model-00023-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95e3e7860bf1707bd6210f89668de0088656de854064cba392e1b3a09ce6c38f
+size 4882269832

model-00024-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2431ef5c6f659465a319c7ec4a9baadf336233825b24d15801f7264fe1fb8c60
+size 4831889152

model-00025-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f00ea4d8786f3760a62861005bea991ffe251ea947e7d6ae5dffbe51bf35674e
+size 4831938536

model-00026-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9050f69468aa70de6960ae8649d05c1e9a1f89278b71b6db23e7906aa46eb04c
+size 4831938544

model-00027-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:250f32fbdc0e91febef87967b2366b8d4a64eaca4c8fa78f526b3c770fb1879f
+size 4882269832

model-00028-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aca514c2dfee913ff24f22c4a32fa2a6865c733a59152cd3e7692fad0079a636
+size 4831889152

model-00029-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2dbb9cb132346a85a497ab8702e4569ee8c13af61187ebfa31b9ff4fadff15a1
+size 4831938536

model-00030-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e73e38bfa9725d62d2695a0a5e8664f5fb5f73e15cdbc4d844ac0e356b448c1e
+size 4831938544

model-00031-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cf11153ed1848261fa376e9e376c8fa7ec1ef1567f47ad5b5251d1848dfcb2a
+size 4882269832

model-00032-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83bfd8ec76d3670dc0953e0f53c2f68b2bf67b35ccb3e419f9b0cce7bb5143e8
+size 4831889152

model-00033-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6c34f13613183c5a661c59fa028fc3d3ab17aa48cb60b5e194ae3b60f8bc5ea2
+size 4831938536

model-00034-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c81c40c70235e9144980e1c00eb8b41cc53abf95f9704d6a054e6ff28b75aa27
+size 4831938544

model-00035-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bba77e48b556448202274bc884aebb6e3590d05404327abec2dd6c2946c1287
+size 4882269832

model-00036-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e42cca3bf45b90ed19ec9d831d21fc873b760a14c42a1d114193821bdb815582
+size 4831889152

model-00037-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cad620502c0225380226da0460ad43bd23525bdaf2d6198dbd4a65adefcadd78
+size 4831938536

model-00038-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e5bc554a36b126feb4504582dde086a4e0838999e9aba790b30752e3e6c5e04
+size 4831938544

model-00039-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0a262829fc4024fe6e0c5e6e058e3ef791ac7034f752f68f54ccf2e2ae49cf70
+size 4882269832

model-00040-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f484ebf0b11db87df154d91eb55dcb5d2e804ddc0075380b0e9f5a6faaec2831
+size 4831889152

model-00041-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a143645a027ba67eb10142992b388106a547d9a060aa308f7a1c2f46e17f36fa
+size 4831938536

model-00042-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8db19885a39556abea9c6a120168b0b4a90788923685d8e83f6a5bb4adcd127
+size 4831938544

model-00043-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df76108a2fdfb7bdc32a319e41e17d5311d3d4843b51ae57be6c8d04e71fea94
+size 4882269832

model-00044-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97b1cd99245ca0435234483dee869c9f9e180fe97eb1337b26278023e3713e71
+size 4831889152

model-00045-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f223e2bd4f08b7ea7286185d49b7d72ed644d672192638d35ec3edeab445ee1b
+size 4831938536

model-00046-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aec06baea96de09154cfa44a2bbfd1f839188446e655d83d96f5079b4d4581b0
+size 4831938544

model-00047-of-00051.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9623ea47051d9e0ca0dbb17ada7d1ceacb509a6d71cd4bed28de9543063a8da
+size 4882269832