LibreFine tune on local 4090.
I want to fine-tune LibreFlux. Is it even possible on a 4090 local? Second -- do you have a .json file to use as an example? Nowhere is anything on finetuning with ST.
Yes, you can. Use the LyCORIS configuration here: https://huggingface.co/jimmycarter/LibreFLUX/blob/main/lycoris_config.3090.json
There's a somewhat dated guide here: https://github.com/AmericanPresidentJimmyCarter/simple-flux-lora-training
You just need to add the --flux_attention_masked_training flag and use the model in this repo: https://huggingface.co/jimmycarter/LibreFlux-SimpleTuner
thank you for the reply. miss your post on ST discord sir.
is this correct? I get OOM.
{
"--resume_from_checkpoint": "latest",
"--data_backend_config": "config/multidatabackend.json",
"--aspect_bucket_rounding": 2,
"--seed": 42,
"--minimum_image_size": 0,
"--disable_benchmark": false,
"--output_dir": "output/sophie1XTD",
"--lora_type": "lycoris",
"--lycoris_config": "config/lycoris_config.3090.json",
"--flux_attention_masked_training" : true ,
"--flux_schedule_auto_shift": "false",
"--flux_schedule_shift": 1,
"--flux_guidance_value": 4,
"--report_to": "wandb",
"--max_train_steps": 3500,
"--num_train_epochs": 0,
"--checkpointing_steps": 100,
"--checkpoints_total_limit": 30,
"--model_type": "full",
"--pretrained_model_name_or_path": "jimmycarter/LibreFlux-SimpleTuner",
"--model_family": "flux",
"--train_batch_size": 2,
"--gradient_checkpointing": "true",
"--caption_dropout_probability": 0.0,
"--resolution_type": "pixel_area",
"--resolution": 1024,
"--validation_seed": 42,
"--validation_steps": 100,
"--validation_resolution": "1024x1024",
"--validation_guidance": 3.5,
"--validation_guidance_rescale": "0.0",
"--validation_num_inference_steps": "20",
"--validation_prompt": "A photo-realistic image of a cat.",
"--mixed_precision": "bf16",
"--optimizer": "optimi-lion",
"--learning_rate": "0.0000018",
"--max_grad_norm": "0.1",
"--lr_end": "5e-07",
"--lr_scheduler": "polynomial",
"--lr_warmup_steps": 20,
"--validation_torch_compile": "false"
}