How to continue pt / sft on this model，any suggestions?
#53
by
Ken0102030405
- opened
I wanna fit the model into domin area, how to inject my domin knowledge，any suggestions?
can I use LoRA in sft ??
If your data does not contain CoT, the fine-tuned results may disrupt CoT, causing the <think> tag to disappear. I am currently facing this issue and am unsure how to resolve it.
If your data does not contain CoT, the fine-tuned results may disrupt CoT, causing the
<think>tag to disappear. I am currently facing this issue and am unsure how to resolve it.
I haven't tested on this model., while I tried to sft deepseek_distill with CoT content wrapped by tag, the result seems good.
I working towards the same, but my dataset is not ready yet. I plan to do the following, based on suggestions I've picked up from rombo dawg and others:
- Do SFT against the base model (which in this case is Qwen/Qwen2.5-32B)
- Merge the your SFT model with QwQ-32B with mergekit (rombodawg uses the TIES method and created a writeup here https://www.reddit.com/r/ArtificialInteligence/comments/1g6kkog/continuous_finetuning_working_well_as_expected/)