metadata
library_name: transformers
license: llama3.2
base_model: meta-llama/Llama-3.2-1B-Instruct
tags:
- generated_from_trainer
datasets:
- cc100
model-index:
- name: cc100-zh-Hans-unsupervised-20241110-165558
results: []
cc100-zh-Hans-unsupervised-20241110-165558
This model is a fine-tuned version of meta-llama/Llama-3.2-1B-Instruct on the cc100 dataset. It achieves the following results on the evaluation set:
- Loss: 3.5122
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 24
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 3000
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.3331 | 0.0833 | 250 | 4.3223 |
| 4.07 | 0.1667 | 500 | 4.1778 |
| 3.8022 | 0.25 | 750 | 4.1002 |
| 4.2938 | 0.3333 | 1000 | 3.9540 |
| 4.1352 | 0.4167 | 1250 | 3.9815 |
| 2.9199 | 0.5 | 1500 | 3.8021 |
| 4.3933 | 0.5833 | 1750 | 3.7528 |
| 3.6174 | 0.6667 | 2000 | 3.6856 |
| 3.5882 | 0.75 | 2250 | 3.6198 |
| 3.847 | 0.8333 | 2500 | 3.5746 |
| 3.5199 | 0.9167 | 2750 | 3.5320 |
| 3.735 | 1.0 | 3000 | 3.5122 |
Framework versions
- Transformers 4.45.2
- Pytorch 2.4.1
- Datasets 2.19.1
- Tokenizers 0.20.1