Qwen

/

QwQ-32B

like 2.64k

Follow

Qwen 24.7k

Hey Qwen Team, Great work as usual. It's a beast.

I'm planning to fine-tune the QWQ-32B model on a custom domain dataset and would appreciate some guidance from those with experience.

My Current Situation:
I have a dataset in Alpaca format

I'm unsure about the optimal fine-tuning approach for QWQ-32B

I do have few questions

• Can QWQ-32B be effectively fine-tuned using the Alpaca format dataset, or would this be suboptimal?
• Should I convert my data to use the format instead using DeepSeek or Claude?
• Does QWQ-32B support QLoRA fine-tuning, or is full fine-tuning required?

Can you guide on these things. Yes, I would use QWQ template, but I am skeptical about whether I can still fine-tune with Alpaca format dataset with that template? I want the tag during inference for sure.

Thank you in advance for any insights!