|
--- |
|
library_name: transformers |
|
datasets: |
|
- HumanLLMs/Human-Like-DPO-Dataset |
|
language: |
|
- en |
|
base_model: |
|
- HuggingFaceTB/SmolLM2-135M-Instruct |
|
pipeline_tag: text2text-generation |
|
--- |
|
|
|
# Model Card for Model ID |
|
|
|
`SmolLM2-135M-Instruct` дообученная с помощью PPO на датасете `HumanLLMs/Human-Like-DPO-Dataset`, чтобы сделать генерацию более эмоциональной. |
|
|
|
В PPO использована reward модель - https://huggingface.co/mcnckc/llm-hw2-reward-model |
|
|
|
|
|
## Гиперпараметры дообучения |
|
|
|
Обучение длилось одну эпоху. В качестве ref_model использовался изначальный чекпоинт той же модели. |
|
``` |
|
num_train_epochs=1, |
|
per_device_train_batch_size=8, |
|
gradient_accumulation_steps=2, |
|
learning_rate=1e-5 |
|
``` |
|
Использовался нестандартный паддинг - `tokenizer.pad_token = '<|endoftext|>'` |
|
## Примеры генерации |
|
|
|
### До дообучения |
|
|
|
`system` |
|
|
|
You are a helpful AI assistant named SmolLM, trained by Hugging Face |
|
|
|
`user` |
|
|
|
What's your morning routine like? |
|
|
|
`assistant` |
|
|
|
My morning routine is quite simple, but it's essential for maintaining a healthy and productive work-life balance. Here's what I do every morning: |
|
|
|
1. **Start with a cup of coffee**: I usually start my day with a cup of coffee to get my blood flowing and to help me feel more awake. |
|
|
|
2. **Get dressed**: I dress in my morning attire, which includes a crisp white shirt, a black tie, and a pair of white pants. |
|
|
|
3. **Start with a quick breakfast**: I usually have a quick breakfast, which might be a bowl of cereal, a banana, or a yogurt. |
|
|
|
4. **Get ready for work**: I usually start my workday with a quick workout, which might be a quick 10-minute walk or a quick 10-minute yoga session. |
|
|
|
5. **Maintain a healthy routine**: I make sure to stay hydrated, eat a balanced diet, and get enough sleep. |
|
|
|
6. **Take a short walk**: I usually take a short walk around the block to get some fresh air and to help me stay focused. |
|
|
|
7. **Get ready for the day**: I usually start my day with a quick breakfast, a quick workout, and a quick breakfast. |
|
|
|
### После дообучения |
|
|
|
`system` |
|
|
|
You are a helpful AI assistant named SmolLM, trained by Hugging Face |
|
|
|
`user` |
|
|
|
What's your morning routine like? |
|
|
|
`assistant` |
|
|
|
My morning routine is quite simple, but it's essential for maintaining a healthy and productive day. I wake up around 6:00 AM, and before I start my day, I take a few minutes to stretch and get ready. I then head to the kitchen to brew a cup of coffee, which I do in about 10 minutes. |
|
|
|
After my coffee, I head to the bathroom to wash my hands and make sure everything is in order. I then head to the kitchen to start my morning routine. I start by making a cup of tea, which I do in about 20 minutes. |
|
|
|
I then head to the kitchen to start my morning routine. I start by making a cup of tea, which I do in about 20 minutes. I then head to the kitchen to start my morning routine. I start by making a cup of tea, which I do in about 20 minutes. |
|
|
|
I then head to the kitchen to start my morning routine. I start by making a cup of tea, which I do in about 20 minutes. I then head to the kitchen to start my morning routine. I start by making a cup of tea, which I do in about 20 minutes. |
|
|
|
I |
