license: apache-2.0
base_model: Helsinki-NLP/opus-mt-en-ROMANCE
tags:
- generated_from_trainer
model-index:
- name: seq2seq-finetuned-slang-en
results: []
seq2seq-finetuned-slang-en
This model is a fine-tuned version of Helsinki-NLP/opus-mt-en-ROMANCE on a dataset of 1772 messages in slang English, and manually translated into standard English. There are some typos, since this was a first approach. Thus, insults or acronyms are not taken into account. Also, you may confirm mistakes, such as translating 'ya' almost always as 'yes', while it can also be 'you'. These are nuances I am working on. I am also working on this task for other languages, if you like the project, please contact me.
It achieves the following results on the evaluation set:
- Loss: 0.0286
Model description
More information needed
Intended uses & limitations
This is a prototype. I am working on a better model. The goal would be to finetune a model able to translate slang English into standard English. I am also working on applying it to different languages.
Training and evaluation data
The data partition is as follows: 85% train and val, and 15% for testing.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 3
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.0844 | 1.33 | 500 | 0.0714 |
| 0.0297 | 2.65 | 1000 | 0.0286 |
Framework versions
- Transformers 4.35.2
- Pytorch 2.1.0+cu121
- Tokenizers 0.15.1