sizhkhy/akoul_whitehorseliquidity_25c

llm3br256

This model is a fine-tuned version of meta-llama/Llama-3.2-3B-Instruct on the akoul_whitehorseliquidity_25c dataset. It achieves the following results on the evaluation set:

• Loss: 0.0008

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 25.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.0078	0.0808	25	0.0079
0.0119	0.1616	50	0.0051
0.0036	0.2424	75	0.0032
0.004	0.3232	100	0.0025
0.0019	0.4040	125	0.0020
0.0021	0.4848	150	0.0018
0.0016	0.5657	175	0.0016
0.0013	0.6465	200	0.0015
0.0017	0.7273	225	0.0015
0.0015	0.8081	250	0.0014
0.0023	0.8889	275	0.0013
0.0012	0.9697	300	0.0013
0.0011	1.0505	325	0.0013
0.0011	1.1313	350	0.0013
0.0009	1.2121	375	0.0012
0.0015	1.2929	400	0.0011
0.0025	1.3737	425	0.0011
0.0016	1.4545	450	0.0011
0.001	1.5354	475	0.0011
0.0007	1.6162	500	0.0011
0.0008	1.6970	525	0.0011
0.001	1.7778	550	0.0010
0.0007	1.8586	575	0.0010
0.0013	1.9394	600	0.0009
0.0007	2.0202	625	0.0010
0.0006	2.1010	650	0.0009
0.0007	2.1818	675	0.0009
0.001	2.2626	700	0.0009
0.0015	2.3434	725	0.0009
0.0012	2.4242	750	0.0010
0.0012	2.5051	775	0.0009
0.0015	2.5859	800	0.0010
0.0011	2.6667	825	0.0009
0.0007	2.7475	850	0.0009
0.0009	2.8283	875	0.0009
0.0009	2.9091	900	0.0008
0.001	2.9899	925	0.0009
0.0006	3.0707	950	0.0009
0.0006	3.1515	975	0.0009
0.0007	3.2323	1000	0.0009
0.0004	3.3131	1025	0.0009

Framework versions

• PEFT 0.12.0
• Transformers 4.46.1
• Pytorch 2.4.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.3