Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,19 +1,19 @@
|
|
| 1 |
# DeepSeek-R1-Distill-Llama-8B-q4f16_ft-MLC
|
| 2 |
|
| 3 |
-
|
|
| 4 |
-
|
| 5 |
-
| Source Model
|
| 6 |
-
| Inference API
|
| 7 |
-
| Quantization
|
| 8 |
-
| Model Type
|
| 9 |
-
| Vocab Size
|
| 10 |
-
| Context Window Size
|
| 11 |
-
| Prefill Chunk Size
|
| 12 |
-
| Temperature
|
| 13 |
-
| Repetition Penalty
|
| 14 |
-
| `top_p`
|
| 15 |
-
| `pad_token_id`
|
| 16 |
-
| `bos_token_id`
|
| 17 |
-
| `eos_token_id`
|
| 18 |
|
| 19 |
See [`jetson-ai-lab.com/models.html`](https://jetson-ai-lab.com/models.html) for benchmarks, examples, and containers to deploy local serving and inference for these quantized models.
|
|
|
|
| 1 |
# DeepSeek-R1-Distill-Llama-8B-q4f16_ft-MLC
|
| 2 |
|
| 3 |
+
| | Model Configuration |
|
| 4 |
+
|---------------------|:-------------------------------------------------------------------------------------------------------------:|
|
| 5 |
+
| Source Model | [`deepseek-ai/DeepSeek-R1-Distill-Llama-8B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B) |
|
| 6 |
+
| Inference API | `MLC_LLM` |
|
| 7 |
+
| Quantization | `q4f16_ft` |
|
| 8 |
+
| Model Type | `llama` |
|
| 9 |
+
| Vocab Size | `128256` |
|
| 10 |
+
| Context Window Size | `131072` |
|
| 11 |
+
| Prefill Chunk Size | `8192` |
|
| 12 |
+
| Temperature | `0.6` |
|
| 13 |
+
| Repetition Penalty | `1.0` |
|
| 14 |
+
| `top_p` | `0.95` |
|
| 15 |
+
| `pad_token_id` | `0` |
|
| 16 |
+
| `bos_token_id` | `128000` |
|
| 17 |
+
| `eos_token_id` | `128001` |
|
| 18 |
|
| 19 |
See [`jetson-ai-lab.com/models.html`](https://jetson-ai-lab.com/models.html) for benchmarks, examples, and containers to deploy local serving and inference for these quantized models.
|