nllb-200-distilled-1.3B-finetuned-finetuned

This model is a fine-tuned version of KevinKibe/nllb-200-distilled-1.3B-finetuned on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7134
• Rouge: 0.467
• Gen Len: 49.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 1000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge	Gen Len
6.4789	100.0	100	5.8836	0.082	32.5
4.8961	200.0	200	4.8026	0.1552	29.0
3.636	300.0	300	3.5860	0.0292	63.0
2.4528	400.0	400	2.4531	0.2018	28.0
1.4112	500.0	500	1.6682	0.1899	70.0
0.6738	600.0	600	1.1858	0.1907	68.0
0.2921	700.0	700	0.8546	0.2776	28.0
0.1361	800.0	800	0.7109	0.3649	59.5
0.0764	900.0	900	0.7293	0.4568	54.0
0.0559	1000.0	1000	0.7134	0.467	49.0

Framework versions

• Transformers 4.39.2
• Pytorch 2.2.2+cu121
• Datasets 2.21.0
• Tokenizers 0.15.2