outputs

This model is a fine-tuned version of unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3463

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 20
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.9219	0.1773	50	0.6719
0.529	0.3546	100	0.4809
0.448	0.5319	150	0.4260
0.4242	0.7092	200	0.4043
0.4133	0.8865	250	0.3898
0.3839	1.0638	300	0.3803
0.3767	1.2411	350	0.3735
0.3708	1.4184	400	0.3673
0.3605	1.5957	450	0.3632
0.3721	1.7730	500	0.3593
0.3668	1.9504	550	0.3554
0.3457	2.1277	600	0.3536
0.3352	2.3050	650	0.3518
0.3426	2.4823	700	0.3490
0.3356	2.6596	750	0.3478
0.3376	2.8369	800	0.3463

Framework versions

• PEFT 0.14.0
• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0