|
--- |
|
license: apache-2.0 |
|
base_model: |
|
- Qwen/QwQ-32B |
|
--- |
|
# Like my work? Support me on patreon for only $5 a month and get to vote on what model's i make next as well as get access to this org's private repo's |
|
Subscribe bellow: |
|
- Patreon.com/Rombodawg |
|
__________________________________________________ |
|
|
|
# Rombo-LLM-V3.1-QWQ-32b |
|
|
|
 |
|
|
|
Rombo-LLM-V3.1-QWQ-32b is a Continued Finetune model (Merge only) of (Qwen/QwQ-32B) and its base model (Qwen/Qwen2.5-32B). This merge is done to decrease catastrophic forgetting during finetuning, and increase overall performance of the model. The tokenizers are taken from the QwQ-32B for thinking capabilities. |
|
|
|
You can find the Continous Finetuning Doc bellow for more details on the full method: |
|
|
|
- https://docs.google.com/document/d/1OjbjU5AOz4Ftn9xHQrX3oFQGhQ6RDUuXQipnQ9gn6tU/edit?usp=sharing |
|
|
|
Quantized model: (coming soon) |
|
|
|
- |
|
|
|
Benchmarks: (coming soon) |
|
|
