metadata
license: other
base_model: Qwen/Qwen1.5-14B-Chat
tags:
- llama-factory
- full
- generated_from_trainer
model-index:
- name: PsycoLLM
results: []
PsycoLLM 心理大模型
This model is a fine-tuned version of Qwen/Qwen1.5-14B-Chat on the QAs, the ds and the dialogue datasets. It achieves the following results on the evaluation set:
- Loss: 1.3823
Model description
We will open source the entire dataset in the future. Please keep focusing on our work.
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- distributed_type: multi-GPU
- num_devices: 8
- gradient_accumulation_steps: 16
- total_train_batch_size: 128
- total_eval_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 20
- num_epochs: 3.0
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 1.2867 | 0.63 | 100 | 1.2870 |
| 0.9624 | 1.27 | 200 | 1.2869 |
| 0.9492 | 1.9 | 300 | 1.2718 |
| 0.6774 | 2.53 | 400 | 1.3823 |
Framework versions
- Transformers 4.39.1
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2