|
--- |
|
base_model: genmo/mochi-1-preview |
|
library_name: diffusers |
|
license: apache-2.0 |
|
widget: [] |
|
tags: |
|
- text-to-video |
|
- diffusers-training |
|
- diffusers |
|
- lora |
|
- mochi-1-preview |
|
- mochi-1-preview-diffusers |
|
- template:sd-lora |
|
- text-to-video |
|
- diffusers-training |
|
- diffusers |
|
- lora |
|
- mochi-1-preview |
|
- mochi-1-preview-diffusers |
|
- template:sd-lora |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the training script had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
|
|
# Mochi-1 Preview LoRA Finetune |
|
|
|
<Gallery /> |
|
|
|
## Model description |
|
|
|
This is a lora finetune of the Mochi-1 preview model `genmo/mochi-1-preview`. |
|
|
|
The model was trained using [CogVideoX Factory](https://github.com/a-r-r-o-w/cogvideox-factory) - a repository containing memory-optimized training scripts for the CogVideoX and Mochi family of models using [TorchAO](https://github.com/pytorch/ao) and [DeepSpeed](https://github.com/microsoft/DeepSpeed). The scripts were adopted from [CogVideoX Diffusers trainer](https://github.com/huggingface/diffusers/blob/main/examples/cogvideo/train_cogvideox_lora.py). |
|
|
|
## Download model |
|
|
|
[Download LoRA](soumildatta/mochi-lora/tree/main) in the Files & Versions tab. |
|
|
|
## Usage |
|
|
|
Requires the [🧨 Diffusers library](https://github.com/huggingface/diffusers) installed. |
|
|
|
```py |
|
from diffusers import MochiPipeline |
|
from diffusers.utils import export_to_video |
|
import torch |
|
|
|
pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview") |
|
pipe.load_lora_weights("CHANGE_ME") |
|
pipe.enable_model_cpu_offload() |
|
|
|
with torch.autocast("cuda", torch.bfloat16): |
|
video = pipe( |
|
prompt="CHANGE_ME", |
|
guidance_scale=6.0, |
|
num_inference_steps=64, |
|
height=480, |
|
width=848, |
|
max_sequence_length=256, |
|
output_type="np" |
|
).frames[0] |
|
export_to_video(video) |
|
``` |
|
|
|
For more details, including weighting, merging and fusing LoRAs, check the [documentation](https://huggingface.co/docs/diffusers/main/en/using-diffusers/loading_adapters) on loading LoRAs in diffusers. |
|
|
|
|
|
|
|
## Intended uses & limitations |
|
|
|
#### How to use |
|
|
|
```python |
|
# TODO: add an example code snippet for running this diffusion pipeline |
|
``` |
|
|
|
#### Limitations and bias |
|
|
|
[TODO: provide examples of latent issues and potential remediations] |
|
|
|
## Training details |
|
|
|
[TODO: describe the data used to train the model] |
