metadata

license: llama2
base_model: meta-llama/Llama-2-7b-hf
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: lmind_hotpot_train8000_eval7405_v1_qa_5e-5_lora2
    results: []

lmind_hotpot_train8000_eval7405_v1_qa_5e-5_lora2

This model is a fine-tuned version of meta-llama/Llama-2-7b-hf on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 3.2298
Accuracy: 0.5839

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 2
eval_batch_size: 2
seed: 42
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 4
total_train_batch_size: 32
total_eval_batch_size: 8
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: constant
lr_scheduler_warmup_ratio: 0.05
num_epochs: 20.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.798	1.0	250	1.8213	0.6067
1.7	2.0	500	1.8046	0.6077
1.5869	3.0	750	1.8293	0.6071
1.4349	4.0	1000	1.8974	0.6043
1.3111	5.0	1250	1.9769	0.6015
1.197	6.0	1500	2.0635	0.5992
1.0729	7.0	1750	2.1523	0.5975
0.9833	8.0	2000	2.2640	0.5947
0.8672	9.0	2250	2.3643	0.5924
0.7883	10.0	2500	2.4598	0.5908
0.6879	11.0	2750	2.5669	0.5890
0.6295	12.0	3000	2.7000	0.5885
0.5545	13.0	3250	2.8281	0.5851
0.5208	14.0	3500	2.8794	0.5853
0.4679	15.0	3750	2.9184	0.5863
0.4464	16.0	4000	3.0791	0.5852
0.4136	17.0	4250	3.0832	0.5856
0.4021	18.0	4500	3.0944	0.5847
0.3776	19.0	4750	3.2120	0.5828
0.373	20.0	5000	3.2298	0.5839

Framework versions

Transformers 4.34.0
Pytorch 2.1.0+cu121
Datasets 2.18.0
Tokenizers 0.14.1