seq2seq-finetuned-slang-en

This model is a fine-tuned version of Helsinki-NLP/opus-mt-en-ROMANCE on a dataset of 1772 messages in slang English, and manually translated into standard English. There are some typos, since this was a first approach. Thus, insults or acronyms are not taken into account. Also, you may confirm mistakes, such as translating 'ya' almost always as 'yes', while it can also be 'you'. These are nuances I am working on. I am also working on this task for other languages, if you like the project, please contact me.

It achieves the following results on the evaluation set:

• Loss: 0.0286

Model description

More information needed

Intended uses & limitations

This is a prototype. I am working on a better model. The goal would be to finetune a model able to translate slang English into standard English. I am also working on applying it to different languages.

Training and evaluation data

The data partition is as follows: 85% train and val, and 15% for testing.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.0844	1.33	500	0.0714
0.0297	2.65	1000	0.0286

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Tokenizers 0.15.1