metadata

library_name: transformers
datasets:
  - HuggingFaceH4/ultrachat_200k
base_model: google/gemma-2b
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms

Gemma 2B Zephyr SFT

The Zephyr SFT recipe applied on top of Gemma 2B

Model description

Model type: A 8.5B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
Language(s) (NLP): Primarily English
Finetuned from model: google/gemma-7b

Recipe

We trained using the alignment handbook recipe and logging to W&B

Visit the W&B workspace here

License

This model has the same license as the original Gemma model collection

Compute provided by Lambda Labs - 8xA100 80GB node

Around 2 hours to train