phi-3-mini-sft-indicqa-hindi-v0.1

model description

this model is a fine-tuned version of microsoft/Phi-3-mini-128k-instruct on the dataset ai4bharat-IndicQA-hi-202410.

prepared by @sepiatone.

intended uses & limitations

intended for an educational and non-commercial purpose.

training procedure

training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0002
  • train_batch_size: 4
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.03
  • num_epochs: 1
  • mixed_precision_training: Native AMP

library versions

  • PEFT 0.13.2
  • Transformers 4.44.2
  • Pytorch 2.5.0+cu121
  • Datasets 3.0.2
  • Tokenizers 0.19.1
Downloads last month
10
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.

Model tree for sepiatone/phi-3-mini-sft-indicqa-hi

Adapter
(440)
this model

Datasets used to train sepiatone/phi-3-mini-sft-indicqa-hi