---
license: apache-2.0
tags:
- generated_from_trainer
- google/gemma
- PyTorch
- transformers
- trl
- peft
- tensorboard
base_model: google/gemma-2b
widget:
  - example_title: Pirate!
    messages:
      - role: system
        content: You are a pirate chatbot who always responds with Arr!
      - role: user
        content: "There's a llama on my lawn, how can I get rid of him?"
    output:
      text: >-
        Arr! 'Tis a puzzlin' matter, me hearty! A llama on yer lawn be a rare
        sight, but I've got a plan that might help ye get rid of 'im. Ye'll need
        to gather some carrots and hay, and then lure the llama away with the
        promise of a tasty treat. Once he's gone, ye can clean up yer lawn and
        enjoy the peace and quiet once again. But beware, me hearty, for there
        may be more llamas where that one came from! Arr!
model-index:
- name: gemma-2b-chat
  results: []
datasets:
- HuggingFaceH4/deita-10k-v0-sft
language:
- en
pipeline_tag: text-generation
---

# Model Card for gemma-2b-chat:

**gemma-2b-chat** is a language model that is trained to act as helpful assistant. It is a finetuned version of [google/gemma-2b](https://huggingface.co/google/gemma-2b) that was trained using `SFTTrainer` on publicly available dataset [
HuggingFaceH4/deita-10k-v0-sft](https://huggingface.co/datasets/HuggingFaceH4/deita-10k-v0-sft).

## Training Procedure:

The training code used to create this model was generated by [Menouar/LLM-FineTuning-Notebook-Generator](https://huggingface.co/spaces/Menouar/LLM-FineTuning-Notebook-Generator).



## Training hyperparameters

The following hyperparameters were used during the training:


- output_dir: temp_gemma-2b-chat
 
- overwrite_output_dir: True
 
- do_train: False
 
- do_eval: False
 
- do_predict: False
 
- evaluation_strategy: no
 
- prediction_loss_only: False
 
- per_device_train_batch_size: 3
 
- per_device_eval_batch_size: 8
 
- per_gpu_train_batch_size: None
 
- per_gpu_eval_batch_size: None
 
- gradient_accumulation_steps: 2
 
- eval_accumulation_steps: None
 
- eval_delay: 0
 
- learning_rate: 2e-05
 
- weight_decay: 0.0
 
- adam_beta1: 0.9
 
- adam_beta2: 0.999
 
- adam_epsilon: 1e-08
 
- max_grad_norm: 0.3
 
- num_train_epochs: 1
 
- max_steps: -1
 
- lr_scheduler_type: cosine
 
- lr_scheduler_kwargs: {}
 
- warmup_ratio: 0.1
 
- warmup_steps: 0
 
- log_level: passive
 
- log_level_replica: warning
 
- log_on_each_node: True
 
- logging_dir: temp_gemma-2b-chat/runs/Mar11_17-14-25_f4965e0005f4
 
- logging_strategy: steps
 
- logging_first_step: False
 
- logging_steps: 10
 
- logging_nan_inf_filter: True
 
- save_strategy: epoch
 
- save_steps: 500
 
- save_total_limit: None
 
- save_safetensors: True
 
- save_on_each_node: False
 
- save_only_model: False
 
- no_cuda: False
 
- use_cpu: False
 
- use_mps_device: False
 
- seed: 42
 
- data_seed: None
 
- jit_mode_eval: False
 
- use_ipex: False
 
- bf16: True
 
- fp16: False
 
- fp16_opt_level: O1
 
- half_precision_backend: auto
 
- bf16_full_eval: False
 
- fp16_full_eval: False
 
- tf32: None
 
- local_rank: 0
 
- ddp_backend: None
 
- tpu_num_cores: None
 
- tpu_metrics_debug: False
 
- debug: []
 
- dataloader_drop_last: False
 
- eval_steps: None
 
- dataloader_num_workers: 0
 
- dataloader_prefetch_factor: None
 
- past_index: -1
 
- run_name: temp_gemma-2b-chat
 
- disable_tqdm: False
 
- remove_unused_columns: True
 
- label_names: None
 
- load_best_model_at_end: False
 
- metric_for_best_model: None
 
- greater_is_better: None
 
- ignore_data_skip: False
 
- fsdp: []
 
- fsdp_min_num_params: 0
 
- fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
 
- fsdp_transformer_layer_cls_to_wrap: None
 
- accelerator_config: AcceleratorConfig(split_batches=False, dispatch_batches=None, even_batches=True, use_seedable_sampler=True)
 
- deepspeed: None
 
- label_smoothing_factor: 0.0
 
- optim: adamw_torch_fused
 
- optim_args: None
 
- adafactor: False
 
- group_by_length: False
 
- length_column_name: length
 
- report_to: ['tensorboard']
 
- ddp_find_unused_parameters: None
 
- ddp_bucket_cap_mb: None
 
- ddp_broadcast_buffers: None
 
- dataloader_pin_memory: True
 
- dataloader_persistent_workers: False
 
- skip_memory_metrics: True
 
- use_legacy_prediction_loop: False
 
- push_to_hub: False
 
- resume_from_checkpoint: None
 
- hub_model_id: None
 
- hub_strategy: every_save
 
- hub_token: None
 
- hub_private_repo: False
 
- hub_always_push: False
 
- gradient_checkpointing: True
 
- gradient_checkpointing_kwargs: {'use_reentrant': False}
 
- include_inputs_for_metrics: False
 
- fp16_backend: auto
 
- push_to_hub_model_id: None
 
- push_to_hub_organization: None
 
- push_to_hub_token: None
 
- mp_parameters: 
 
- auto_find_batch_size: False
 
- full_determinism: False
 
- torchdynamo: None
 
- ray_scope: last
 
- ddp_timeout: 1800
 
- torch_compile: False
 
- torch_compile_backend: None
 
- torch_compile_mode: None
 
- dispatch_batches: None
 
- split_batches: None
 
- include_tokens_per_second: False
 
- include_num_input_tokens_seen: False
 
- neftune_noise_alpha: None
 
- distributed_state: Distributed environment: NO
Num processes: 1
Process index: 0
Local process index: 0
Device: cuda

 
- _n_gpu: 1
 
- __cached__setup_devices: cuda:0
 
- deepspeed_plugin: None