qwen2.5-0.5b-expo-DPO-EXPERIMENT

This model is a fine-tuned version of hZzy/qwen2.5-0.5b-sft-news-IFT on the hZzy/train_pairwise dataset. It achieves the following results on the evaluation set:

• Loss: 0.6818
• Logps: -92.0550
• Logits: -1.5636
• Objective: 0.6891
• Dpo Loss: 0.6891
• Regularize: 0.6891
• Ranking Simple: 0.5196
• Ranking Idealized: 0.5888
• Ranking Idealized Expo: 0.5103

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-07
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 6
• gradient_accumulation_steps: 12
• total_train_batch_size: 288
• total_eval_batch_size: 24
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss	Logps	Logits	Objective	Dpo Loss	Regularize	Ranking Simple	Ranking Idealized	Ranking Idealized Expo
0.6855	0.2834	50	0.6889	-90.7669	-1.4343	0.6918	0.6918	0.6918	0.5103	0.5888	0.5103
0.6746	0.5668	100	0.6858	-90.9748	-1.4764	0.6899	0.6899	0.6899	0.5093	0.5888	0.5103
0.6601	0.8503	150	0.6828	-90.8063	-1.5179	0.6886	0.6886	0.6886	0.5134	0.5888	0.5103
0.6473	1.1337	200	0.6826	-91.9779	-1.5427	0.6890	0.6890	0.6890	0.5176	0.5888	0.5103
0.6449	1.4171	250	0.6813	-91.6044	-1.5537	0.6887	0.6887	0.6887	0.5176	0.5888	0.5103
0.6384	1.7005	300	0.6818	-92.0140	-1.5627	0.6890	0.6890	0.6890	0.5186	0.5888	0.5103
0.6431	1.9839	350	0.6818	-92.0550	-1.5636	0.6891	0.6891	0.6891	0.5196	0.5888	0.5103

Framework versions

• Transformers 4.42.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1