run-gemma

This model is a fine-tuned version of google/gemma-2b on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4238
• Accuracy: 0.7778

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 20
• training_steps: 200

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
0.7031	0.1087	10	0.6740	0.5752
0.6719	0.2174	20	0.6701	0.6046
0.6602	0.3261	30	0.6497	0.6601
0.6367	0.4348	40	0.6113	0.7059
0.6172	0.5435	50	0.5686	0.7320
0.5625	0.6522	60	0.5276	0.7451
0.5664	0.7609	70	0.4938	0.7451
0.5859	0.8696	80	0.4651	0.7712
0.5	0.9783	90	0.4560	0.7647
0.5898	1.0870	100	0.4560	0.7582
0.5664	1.1957	110	0.4459	0.7516
0.4648	1.3043	120	0.4387	0.7745
0.5117	1.4130	130	0.4306	0.7712
0.4219	1.5217	140	0.4239	0.7680
0.3828	1.6304	150	0.4314	0.7680
0.3789	1.7391	160	0.4319	0.7647
0.3828	1.8478	170	0.4291	0.7680
0.3438	1.9565	180	0.4279	0.7745
0.3496	2.0652	190	0.4274	0.7712
0.3945	2.1739	200	0.4238	0.7778

Framework versions

• PEFT 0.12.0
• Transformers 4.44.0
• Pytorch 2.4.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1