wandb: WARNING Calling wandb.login() after wandb.init() has no effect.
2024-05-26 01:46:36 (UTC) - 0:00:04 - finetune.wrapped_model - INFO - Reloading model from /root/mistral_models/7B-v0.3/consolidated.safetensors ...
2024-05-26 01:46:36 (UTC) - 0:00:04 - finetune.wrapped_model - INFO - Converting model to dtype torch.bfloat16 ...
2024-05-26 01:46:36 (UTC) - 0:00:04 - finetune.wrapped_model - INFO - Loaded model on cpu!
2024-05-26 01:46:36 (UTC) - 0:00:04 - finetune.wrapped_model - INFO - Initializing lora layers ...
2024-05-26 01:46:36 (UTC) - 0:00:05 - finetune.wrapped_model - INFO - Finished initialization!
2024-05-26 01:46:36 (UTC) - 0:00:05 - finetune.wrapped_model - INFO - Sharding model over 1 GPUs ...
2024-05-26 01:46:38 (UTC) - 0:00:06 - finetune.wrapped_model - INFO - Model sharded!
2024-05-26 01:46:38 (UTC) - 0:00:06 - finetune.wrapped_model - INFO - 167,772,160 out of 7,415,795,712 parameter are finetuned (2.26%).