math_question_grade_detection_v12-15-24_v2

This model is a fine-tuned version of allenai/scibert_scivocab_uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7833
• Accuracy: 0.8156
• Precision: 0.8202
• Recall: 0.8156
• F1: 0.8150

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 5000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
No log	0.0855	50	3.7362	0.0451	0.0376	0.0451	0.0314
No log	0.1709	100	3.5268	0.1037	0.0640	0.1037	0.0240
No log	0.2564	150	3.2696	0.1374	0.0502	0.1374	0.0535
No log	0.3419	200	3.0470	0.1748	0.1029	0.1748	0.0859
No log	0.4274	250	2.8364	0.2113	0.1981	0.2113	0.1345
No log	0.5128	300	2.5545	0.3631	0.3303	0.3631	0.2989
No log	0.5983	350	2.3118	0.4025	0.3901	0.4025	0.3347
No log	0.6838	400	2.1470	0.4207	0.4281	0.4207	0.3816
No log	0.7692	450	1.9231	0.4784	0.4605	0.4784	0.4348
2.8424	0.8547	500	1.7209	0.5235	0.4724	0.5235	0.4737
2.8424	0.9402	550	1.6283	0.5370	0.5227	0.5370	0.5033
2.8424	1.0256	600	1.5094	0.5648	0.5333	0.5648	0.5175
2.8424	1.1111	650	1.4568	0.5783	0.5770	0.5783	0.5429
2.8424	1.1966	700	1.4039	0.5821	0.5773	0.5821	0.5475
2.8424	1.2821	750	1.2911	0.6244	0.6001	0.6244	0.5936
2.8424	1.3675	800	1.2526	0.6263	0.5988	0.6263	0.5948
2.8424	1.4530	850	1.2468	0.6427	0.6255	0.6427	0.6183
2.8424	1.5385	900	1.1799	0.6427	0.6107	0.6427	0.6182
2.8424	1.6239	950	1.1391	0.6695	0.6347	0.6695	0.6395
1.3882	1.7094	1000	1.1030	0.6571	0.6434	0.6571	0.6253
1.3882	1.7949	1050	1.1132	0.6580	0.6580	0.6580	0.6328
1.3882	1.8803	1100	1.0407	0.6888	0.6613	0.6888	0.6648
1.3882	1.9658	1150	1.0152	0.6840	0.6513	0.6840	0.6563
1.3882	2.0513	1200	0.9474	0.7089	0.6872	0.7089	0.6870
1.3882	2.1368	1250	0.9445	0.7185	0.7093	0.7185	0.7004
1.3882	2.2222	1300	0.9189	0.7147	0.7058	0.7147	0.7004
1.3882	2.3077	1350	0.9597	0.7137	0.7012	0.7137	0.6969
1.3882	2.3932	1400	0.9233	0.7214	0.7081	0.7214	0.7062
1.3882	2.4786	1450	0.9196	0.7214	0.7270	0.7214	0.7017
0.8711	2.5641	1500	0.9337	0.7070	0.7023	0.7070	0.6888
0.8711	2.6496	1550	0.8891	0.7157	0.7171	0.7157	0.7064
0.8711	2.7350	1600	0.8567	0.7387	0.7400	0.7387	0.7279
0.8711	2.8205	1650	0.8473	0.7358	0.7364	0.7358	0.7237
0.8711	2.9060	1700	0.8563	0.7387	0.7335	0.7387	0.7289
0.8711	2.9915	1750	0.8892	0.7262	0.7270	0.7262	0.7141
0.8711	3.0769	1800	0.8978	0.7253	0.7286	0.7253	0.7148
0.8711	3.1624	1850	0.8036	0.7570	0.7526	0.7570	0.7466
0.8711	3.2479	1900	0.8240	0.7570	0.7602	0.7570	0.7489
0.8711	3.3333	1950	0.8134	0.7560	0.7540	0.7560	0.7478
0.6038	3.4188	2000	0.7952	0.7618	0.7652	0.7618	0.7542
0.6038	3.5043	2050	0.7927	0.7618	0.7591	0.7618	0.7536
0.6038	3.5897	2100	0.8155	0.7445	0.7520	0.7445	0.7349
0.6038	3.6752	2150	0.7881	0.7598	0.7659	0.7598	0.7523
0.6038	3.7607	2200	0.7905	0.7560	0.7639	0.7560	0.7461
0.6038	3.8462	2250	0.8357	0.7598	0.7666	0.7598	0.7531
0.6038	3.9316	2300	0.7636	0.7733	0.7696	0.7733	0.7662
0.6038	4.0171	2350	0.7556	0.7781	0.7816	0.7781	0.7712
0.6038	4.1026	2400	0.7696	0.7704	0.7795	0.7704	0.7664
0.6038	4.1880	2450	0.7992	0.7733	0.7811	0.7733	0.7676
0.3965	4.2735	2500	0.7492	0.7733	0.7679	0.7733	0.7657
0.3965	4.3590	2550	0.7900	0.7695	0.7725	0.7695	0.7649
0.3965	4.4444	2600	0.7793	0.7733	0.7815	0.7733	0.7679
0.3965	4.5299	2650	0.7863	0.7771	0.7799	0.7771	0.7720
0.3965	4.6154	2700	0.8007	0.7723	0.7793	0.7723	0.7662
0.3965	4.7009	2750	0.7483	0.7829	0.7875	0.7829	0.7791
0.3965	4.7863	2800	0.7696	0.7848	0.7905	0.7848	0.7785
0.3965	4.8718	2850	0.7667	0.7848	0.7956	0.7848	0.7809
0.3965	4.9573	2900	0.7565	0.7848	0.7871	0.7848	0.7793
0.3965	5.0427	2950	0.7566	0.7877	0.7890	0.7877	0.7840
0.2703	5.1282	3000	0.7573	0.7839	0.7876	0.7839	0.7802
0.2703	5.2137	3050	0.7797	0.7685	0.7719	0.7685	0.7644
0.2703	5.2991	3100	0.7606	0.7791	0.7817	0.7791	0.7753
0.2703	5.3846	3150	0.7584	0.7867	0.7857	0.7867	0.7815
0.2703	5.4701	3200	0.7527	0.7887	0.7910	0.7887	0.7852
0.2703	5.5556	3250	0.7797	0.7877	0.7957	0.7877	0.7844
0.2703	5.6410	3300	0.8015	0.7800	0.7824	0.7800	0.7739
0.2703	5.7265	3350	0.7943	0.7858	0.7882	0.7858	0.7810
0.2703	5.8120	3400	0.7874	0.7915	0.7931	0.7915	0.7870
0.2703	5.8974	3450	0.7851	0.7925	0.7985	0.7925	0.7897
0.1707	5.9829	3500	0.7383	0.8069	0.8140	0.8069	0.8067
0.1707	6.0684	3550	0.7344	0.8060	0.8101	0.8060	0.8036
0.1707	6.1538	3600	0.7723	0.7935	0.7962	0.7935	0.7883
0.1707	6.2393	3650	0.7735	0.7906	0.7919	0.7906	0.7864
0.1707	6.3248	3700	0.7593	0.8031	0.8047	0.8031	0.7987
0.1707	6.4103	3750	0.7781	0.7992	0.8008	0.7992	0.7949
0.1707	6.4957	3800	0.7867	0.8021	0.8026	0.8021	0.7975
0.1707	6.5812	3850	0.7842	0.8002	0.8054	0.8002	0.7973
0.1707	6.6667	3900	0.8035	0.7867	0.7888	0.7867	0.7829
0.1707	6.7521	3950	0.7807	0.8012	0.8023	0.8012	0.7969
0.1045	6.8376	4000	0.7740	0.8021	0.8065	0.8021	0.7989
0.1045	6.9231	4050	0.7801	0.7983	0.8011	0.7983	0.7934
0.1045	7.0085	4100	0.7776	0.8060	0.8111	0.8060	0.8035
0.1045	7.0940	4150	0.7856	0.8002	0.8056	0.8002	0.7973
0.1045	7.1795	4200	0.7664	0.8156	0.8201	0.8156	0.8138
0.1045	7.2650	4250	0.7835	0.8060	0.8062	0.8060	0.8015
0.1045	7.3504	4300	0.7774	0.8031	0.8033	0.8031	0.7994
0.1045	7.4359	4350	0.7855	0.8060	0.8078	0.8060	0.8025
0.1045	7.5214	4400	0.7852	0.8060	0.8073	0.8060	0.8029
0.1045	7.6068	4450	0.7848	0.8021	0.8031	0.8021	0.7991
0.0725	7.6923	4500	0.7882	0.8050	0.8118	0.8050	0.8028
0.0725	7.7778	4550	0.7846	0.8088	0.8149	0.8088	0.8070
0.0725	7.8632	4600	0.7864	0.8136	0.8185	0.8136	0.8121
0.0725	7.9487	4650	0.7845	0.8156	0.8213	0.8156	0.8147
0.0725	8.0342	4700	0.7802	0.8156	0.8207	0.8156	0.8146
0.0725	8.1197	4750	0.7867	0.8184	0.8241	0.8184	0.8174
0.0725	8.2051	4800	0.7861	0.8165	0.8217	0.8165	0.8155
0.0725	8.2906	4850	0.7871	0.8165	0.8217	0.8165	0.8163
0.0725	8.3761	4900	0.7848	0.8175	0.8223	0.8175	0.8171
0.0725	8.4615	4950	0.7835	0.8156	0.8203	0.8156	0.8149
0.0504	8.5470	5000	0.7833	0.8156	0.8202	0.8156	0.8150

Framework versions

• Transformers 4.46.3
• Pytorch 2.4.0
• Datasets 3.1.0
• Tokenizers 0.20.3