File size: 1,244 Bytes
9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d 9410f28 83d3a1d |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
---
library_name: transformers
datasets:
- HuggingFaceH4/ultrachat_200k
base_model: google/gemma-2b
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
---
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/llm_surgery/gemma-zephyr)
# Gemma 2B Zephyr SFT
The [Zephyr](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta) SFT recipe applied on top of Gemma 2B
## Model description
- **Model type:** A 8.5B parameter GPT-like model fine-tuned on a mix of publicly available, synthetic datasets.
- **Language(s) (NLP):** Primarily English
- **Finetuned from model:** [google/gemma-7b](https://huggingface.co/google/gemma-2b)
## Recipe
We trained using the [alignment handbook recipe](https://github.com/huggingface/alignment-handbook/blob/main/scripts/run_sft.py) and logging to W&B
Visit the [W&B workspace here](https://wandb.ai/llm_surgery/gemma-zephyr?nw=nwusercapecape)
## License
This model has the same license as the [original Gemma model collection](https://ai.google.dev/gemma/terms)
## Compute provided by Lambda Labs - 8xA100 80GB node
- Around 2 hours to train
|