Mistral-7B-v0.1_cola_original_cola

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9858
• Accuracy: 0.8692

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 2
• distributed_type: multi-GPU
• num_devices: 6
• gradient_accumulation_steps: 2
• total_train_batch_size: 768
• total_eval_batch_size: 384
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• training_steps: 750

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.0563	2.38	25	1.1091	0.6347
0.7148	4.76	50	0.7231	0.7210
0.5436	7.14	75	0.5349	0.7929
0.4144	9.52	100	0.4578	0.8159
0.3537	11.9	125	0.4236	0.8207
0.3208	14.29	150	0.3996	0.8274
0.3204	16.67	175	0.3874	0.8322
0.2888	19.05	200	0.3939	0.8351
0.2629	21.43	225	0.3837	0.8341
0.2288	23.81	250	0.3904	0.8437
0.1757	26.19	275	0.4016	0.8456
0.1754	28.57	300	0.4165	0.8428
0.1137	30.95	325	0.4513	0.8447
0.0611	33.33	350	0.5193	0.8399
0.0555	35.71	375	0.5863	0.8428
0.0346	38.1	400	0.7369	0.8313
0.0125	40.48	425	0.7936	0.8360
0.0041	42.86	450	0.8821	0.8351
0.0052	45.24	475	0.9493	0.8332
0.0121	47.62	500	1.0594	0.8380
0.0063	50.0	525	1.0706	0.8332
0.0025	52.38	550	1.0518	0.8303
0.0115	54.76	575	1.0344	0.8360
0.0167	57.14	600	1.1477	0.8322
0.0096	59.52	625	1.1863	0.8341
0.004	61.9	650	1.1809	0.8399

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0