metadata

language:
  - en
license: apache-2.0
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - qwen2
  - trl
  - sft
base_model: unsloth/Qwen2-0.5B-Instruct-bnb-4bit
datasets:
  - microsoft/orca-math-word-problems-200k

Coding model comming soon!

Uploaded model

Developed by: NotAiLOL
License: apache-2.0
Finetuned from model : unsloth/Qwen2-0.5B-Instruct-bnb-4bit

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.

Details

This model was trained on microsoft/orca-math-word-problems-200k for 3 epochs with rsLoRA + QLoRA.

Training Loss Graph

The model follows the Alpaca format:

<|im_start|>system
You are a professional mathematician.|im_end|>

<|im_start|>user
{}<|im_end|>

<|im_start|>assistant
{}