qwen2.5-0.5b-expo-DPO-ES-0.1

This model is a fine-tuned version of hZzy/qwen2.5-0.5b-sft-news-IFT on the hZzy/train_pairwise_weighted dataset. It achieves the following results on the evaluation set:

• Loss: 0.6921
• Logps: -91.0309
• Logits: -2.1203
• Objective: 0.6893
• Dpo Loss: 0.6893
• Regularize: 0.6893
• Ranking Simple: 0.5559
• Ranking Idealized: 0.6030
• Ranking Idealized Expo: 0.5223
• Wo Beta: 7.4262

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 3
• gradient_accumulation_steps: 12
• total_train_batch_size: 144
• total_eval_batch_size: 12
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Logps	Logits	Objective	Dpo Loss	Regularize	Ranking Simple	Ranking Idealized	Ranking Idealized Expo	Wo Beta
0.6785	0.1417	50	0.6813	-90.8716	-1.6022	0.6843	0.6843	0.6843	0.5259	0.6030	0.5223	7.8749
0.618	0.2834	100	0.6733	-98.8899	-1.7799	0.6766	0.6766	0.6766	0.5399	0.6030	0.5223	7.7840
0.5667	0.4251	150	0.6866	-99.1230	-1.8072	0.6829	0.6829	0.6829	0.5409	0.6030	0.5223	7.8533
0.5214	0.5668	200	0.6901	-99.5388	-1.8894	0.6904	0.6904	0.6904	0.5445	0.6030	0.5223	7.6995
0.4922	0.7085	250	0.6976	-82.7973	-1.9880	0.6916	0.6916	0.6916	0.5476	0.6030	0.5223	7.8790
0.4535	0.8503	300	0.6921	-91.0309	-2.1203	0.6893	0.6893	0.6893	0.5559	0.6030	0.5223	7.4262
0.423	0.9920	350	0.7057	-88.1615	-1.9880	0.6959	0.6959	0.6959	0.5549	0.6030	0.5223	7.9979
0.2847	1.1337	400	0.7315	-101.6926	-2.0862	0.7281	0.7281	0.7281	0.5424	0.6030	0.5223	8.6326
0.2991	1.2754	450	0.7008	-92.7942	-1.8470	0.6980	0.6980	0.6980	0.5621	0.6030	0.5223	8.2584
0.3065	1.4171	500	0.7180	-96.6747	-2.0065	0.7147	0.7147	0.7147	0.5554	0.6030	0.5223	8.2522
0.2895	1.5588	550	0.7044	-104.2469	-1.8870	0.7077	0.7077	0.7077	0.5652	0.6030	0.5223	8.1947

Framework versions

• Transformers 4.42.0
• Pytorch 2.3.0+cu121
• Datasets 3.2.0
• Tokenizers 0.19.1