Hugging Face's logo

psktoure
/
BERT_BASE_TS_phonetic_wikitext_0.9

Fill-Mask
Transformers
Safetensors
bert
Generated from Trainer
Inference Endpoints
Model card Files Community
BERT_BASE_TS_phonetic_wikitext_0.9 / README.md
psktoure's picture
psktoure
End of training
fa0f33c verified
preview code
|
raw
history blame contribute delete
3.13 kB
metadata
library_name: transformers
license: apache-2.0
base_model: bert-base-uncased
tags:
  - generated_from_trainer
model-index:
  - name: BERT_BASE_TS_phonetic_wikitext_0.9
    results: []

BERT_BASE_TS_phonetic_wikitext_0.9

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.9809

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 256
  • eval_batch_size: 256
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-06
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 10000
  • num_epochs: 10
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
2.2165 0.3019 2000 2.0242
2.1139 0.6039 4000 1.9481
2.0879 0.9058 6000 1.9204
2.0837 1.2077 8000 1.9042
2.0957 1.5097 10000 1.9554
2.1007 1.8116 12000 1.9287
2.1026 2.1135 14000 1.9540
2.1126 2.4155 16000 1.9481
2.1171 2.7174 18000 1.9878
2.1176 3.0193 20000 1.9638
2.1232 3.3213 22000 1.9696
2.1223 3.6232 24000 1.9904
2.1294 3.9251 26000 1.9750
2.127 4.2271 28000 1.9654
2.1342 4.5290 30000 1.9801
2.1402 4.8309 32000 2.0290
2.1344 5.1329 34000 2.0142
2.1373 5.4348 36000 1.9839
2.1325 5.7367 38000 1.9736
2.1367 6.0386 40000 1.9824
2.1417 6.3406 42000 1.9750
2.1319 6.6425 44000 1.9775
2.1387 6.9444 46000 2.0075
2.1359 7.2464 48000 1.9740
2.1315 7.5483 50000 1.9844
2.1388 7.8502 52000 2.0143
2.1302 8.1522 54000 1.9690
2.1368 8.4541 56000 1.9659
2.1347 8.7560 58000 1.9806
2.1304 9.0580 60000 1.9906
2.1268 9.3599 62000 1.9820
2.1241 9.6618 64000 1.9683
2.1308 9.9638 66000 1.9809

Framework versions

  • Transformers 4.45.1
  • Pytorch 2.4.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.20.0