ft_rugec_Ro_vRank1

This model is a fine-tuned version of mika5883/pretrain_rugec_msu on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2420

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 120
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.7554	0.2568	10	0.2400
0.3326	0.5136	20	0.2136
0.268	0.7705	30	0.2006
0.2506	1.0514	40	0.2076
0.1641	1.3082	50	0.2129
0.158	1.5650	60	0.2148
0.1398	1.8218	70	0.2226
0.1217	2.1027	80	0.2195
0.1	2.3596	90	0.2344
0.0864	2.6164	100	0.2318
0.0863	2.8732	110	0.2418
0.0738	3.1541	120	0.2420

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.0.1
• Tokenizers 0.21.0