dpo3

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: nan
• Rewards/chosen: nan
• Rewards/rejected: nan
• Rewards/accuracies: 0.0015
• Rewards/margins: nan
• Logps/rejected: nan
• Logps/chosen: nan
• Logits/rejected: nan
• Logits/chosen: nan

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.07
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 7
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
13488810.24	2.3088	100	nan	nan	nan	0.0015	nan	nan	nan	nan	nan
0.0	4.6176	200	nan	nan	nan	0.0015	nan	nan	nan	nan	nan
0.0	6.9264	300	nan	nan	nan	0.0015	nan	nan	nan	nan	nan

Framework versions

• Transformers 4.45.0
• Pytorch 2.5.1+cu124
• Datasets 2.19.2
• Tokenizers 0.20.3