base

README.md

@@ -0,0 +1,68 @@
+---
+license: apache-2.0
+language:
+  - en
+tags:
+- cogvideox
+- video-generation
+- video-to-video
+- diffusers
+---
+
+# CogvideoX-5b LoRa to control camera movement
+
+The LoRa was trained to control camera movement in 6 directions: `left`, `right`, `up`, `down`, `zoom_in`, `zoom_out`.
+Start prompt with text like this:
+```python
+'Сamera moves to the {}...',
+'Сamera is moving to the {}...',
+'{} camera movement...',
+'{} camera turn...',
+```
+
+### Inference examples
+#### Minimal code example
+```python
+import torch
+from diffusers import CogVideoXImageToVideoPipeline
+from diffusers.utils import export_to_video, load_image
+
+
+pipe = CogVideoXImageToVideoPipeline.from_pretrained(
+    "THUDM/CogVideoX1.5-5B-I2V", torch_dtype=torch.bfloat16
+)
+
+pipe.load_lora_weights("NimVideo/cogvideox1.5-5b-prompt-camera-motion", adapter_name="cogvideox-lora")
+pipe.set_adapters(["cogvideox-lora"], [1.0])
+
+pipe.enable_sequential_cpu_offload()
+pipe.vae.enable_slicing()
+pipe.vae.enable_tiling()
+
+height = 768 
+width = 1360
+image = load_image("resources/car.jpg").resize((width, height))
+prompt = "Camera is moving to the left. A red sports car driving on a winding road."
+
+video_generate = pipe(
+    image=image,
+    prompt=prompt,
+    height=height, 
+    width=width, 
+    num_inference_steps=50,  
+    num_frames=81,  
+    guidance_scale=6.0,
+    generator=torch.Generator().manual_seed(42), 
+).frames[0]
+
+export_to_video(video_generate, output_path, fps=8)
+```
+
+#### Inference with cli and jupyter-notebook examlple you can find <a href="https://github.com/Nim-Video/cogvideox1.5-5b-prompt-camera-motion">on our Github</a>
+
+
+## Acknowledgements
+Original code and models [CogVideoX](https://github.com/THUDM/CogVideo/tree/main).  
+
+## Contacts
+<p>Issues should be raised directly in the repository.</p>

pytorch_lora_weights.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e3fc46d168924f122468bfd8d3dea8b084a8f51b92a3973df54ab8823edc44bf
+size 264286184