README.md

metadata

title: ZeroGPU
emoji: 🖼
colorFrom: purple
colorTo: red
sdk: gradio
sdk_version: 5.25.2
app_file: app.py
pinned: false
license: apache-2.0

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

commands:

download images: python download.py -i 1 -r 2 -o /home/user/app/image_tmp -z

pip install git+https://github.com/huggingface/diffusers

accelerate launch
--deepspeed_config_file ds_config.json
diffusers/examples/dreambooth/train_dreambooth.py
--pretrained_model_name_or_path="runwayml/stable-diffusion-v1-5"
--instance_data_dir="./nyc_ads_dataset"
--instance_prompt="a photo of an urbanad nyc"
--output_dir="./nyc-ad-model"
--resolution=100
--train_batch_size=1
--gradient_accumulation_steps=1
--gradient_checkpointing
--learning_rate=5e-6
--lr_scheduler="constant"
--lr_warmup_steps=0
--max_train_steps=400
--mixed_precision="fp16"
--checkpointing_steps=100
--checkpoints_total_limit=1
--report_to="tensorboard"
--logging_dir="./nyc-ad-model/logs"

fine tune a trained model: --pretrained_model_name_or_path="./nyc-ad-model/checkpoint-400" \

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

pipeline:

1 Fully Fine‑tune image model with ZeRO

accelerate launch --deepspeed_config_file=ds_config_zero3.json train_lora.py fully_fine_tine_stablediffusion

2 SFT 120B OSS 语言模型 with QLoRA

lauguage_model_fine_tuning

3 RLHF PPO 120B OSS 语言模型 with QLoRA : 训练 reward model

lauguage_model_fine_tuning

4 distill 120B OSS模型给20B OSS模型

lauguage_model_fine_tuning 用 Teacher 生成 Response，student模型用LoRA fine tuning

5 Build RAG index embedding table

retrieval_augmented_generation

6 Inference with RAG

inference.py

system flow: input: business or product description text

1. 根据input用RAG取embedding
2. GPT‑OSS 生成 4 个广告文案 + 标题 + 口号（可选语气：专业/活泼/极简）
3. GPT‑OSS 基于选中文案生成 扩展视觉提示词（主体、配色、镜头、艺术风格）
4. stablediffusion model 生成 4 张草图（可选 ControlNet-Layout/Logo 插入）
5. 返回4张海报+后处理 output: an advertisement sentence and post image

design details: LoRA fine tune teacher OSS 120B model using smangrul/ad-copy-generation (广告文案生成) LoRA distill knowledge to OSS 20B model