hZzy/qwen2.5-0.5b-expo-DPO-ES-TRY1

qwen2.5-0.5b-expo-DPO-ES-TRY

This model is a fine-tuned version of hZzy/qwen2.5-0.5b-sft-news-IFT on the hZzy/train_pairwise dataset. It achieves the following results on the evaluation set:

• Loss: 0.6866
• Logps: -91.4116
• Logits: -1.5339
• Objective: 0.6926
• Dpo Loss: 0.6926
• Regularize: 0.6926
• Ranking Simple: 0.5052
• Ranking Idealized: 0.5888
• Ranking Idealized Expo: 0.5093
• Dpo Wo Beta: -0.9551

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 6
• gradient_accumulation_steps: 12
• total_train_batch_size: 288
• total_eval_batch_size: 24
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Dpo Loss	Dpo Wo Beta	Logits	Logps	Validation Loss	Objective	Ranking Idealized	Ranking Idealized Expo	Ranking Simple	Regularize
0.672	0.2041	36	0.6926	-0.9551	-1.5339	-91.4116	0.6866	0.6926	0.5888	0.5093	0.5052	0.6926
0.6533	0.4081	72	0.6885	-1.1473	-1.6311	-92.4758	0.6769	0.6885	0.5888	0.5093	0.5176	0.6885
0.604	0.6122	108	0.6853	-1.3961	-1.7691	-94.0584	0.6773	0.6853	0.5888	0.5093	0.5186	0.6853
0.5911	0.8162	144	0.6886	-1.5153	-1.8165	-95.7578	0.6775	0.6886	0.5888	0.5093	0.5186	0.6886
0.5482	1.0203	180	0.6859	-98.1346	-1.8457	0.6998	0.6998	0.6998	0.5238	0.5888	0.5093	-1.8685
0.5171	1.2244	216	0.6891	-99.3936	-1.8859	0.7004	0.7004	0.7004	0.5248	0.5888	0.5093	-2.0594
0.5093	1.4284	252	0.6999	-102.0847	-1.8968	0.7119	0.7119	0.7119	0.5238	0.5888	0.5093	-2.3165
0.4986	1.6325	288	0.6981	-102.1441	-1.9005	0.7090	0.7090	0.7090	0.5279	0.5888	0.5093	-2.2741
0.5055	1.8366	324	0.6965	-101.5796	-1.8981	0.7080	0.7080	0.7080	0.5279	0.5888	0.5093	-2.2649

Framework versions

• Transformers 4.42.0
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1