Upload folder using huggingface_hub

#5
Valiant Labs org
edited Nov 4

Upload folder using huggingface_hub

Multi commit ID: 1208c825ea8c5c6c6a466784eb521e0f2868de1b80380d406a5430bdeed9fb00

Scheduled commits:

  • Upload 1 file(s) totalling 4.9G (adfacd1fdecad3533c359c30035a150da4a6e0744babe33836ac0fdb654fbd47)
  • Upload 1 file(s) totalling 4.8G (89d39802f0f44aab3aaec9fe367bbde5bdeb1f600dc741fc19855a66bf25179e)
  • Upload 1 file(s) totalling 5.0G (154b6296b96d19d09f03b3a2c445b70c6254c729fee49893e751a6f8bd5f4923)
  • Upload 1 file(s) totalling 5.0G (04e119436f75f623c3d09d0eed91cd5bdcec6c8aed307bdb0a418776a3ed37a3)
  • Upload 1 file(s) totalling 4.8G (aad672a5c32ccddaefef79368d8cca41096ad7c03b3d22377804b1dd0eefa803)
  • Upload 1 file(s) totalling 5.0G (2699bd4029ecae938e19db9fe8e756f6ed9c30214508311656496453decf4e2f)
  • Upload 1 file(s) totalling 2.6G (6609a37293c20b92462f30060c64401cf6deab00569ac44f66f21183915414cb)
  • Upload 7 file(s) totalling 17.3M (c191b597e4415146f4bb0d3df015d2bd960a03ca7ee497671c00fa16ac390ee5)

This is a PR opened using the huggingface_hub library in the context of a multi-commit. PR can be commented as a usual PR. However, please be aware that manually updating the PR description, changing the PR status, or pushing new commits, is not recommended as it might corrupt the commit process. Learn more about multi-commits in this guide.

Valiant Labs org

Multi-commit is now completed! You can ping the repo owner to review the changes. This PR can now be commented or modified without risking to corrupt it.

This is a comment posted using the huggingface_hub library in the context of a multi-commit. Learn more about multi-commits in this guide.

sequelbox changed pull request status to open
sequelbox changed pull request title from [WIP] Upload folder using huggingface_hub (multi-commit 1208c825ea8c5c6c6a466784eb521e0f2868de1b80380d406a5430bdeed9fb00) to Upload folder using huggingface_hub
sequelbox changed pull request status to merged

Is it possible you missed a file? I see part 7 didn't change, which seems extremely unlikely right?

Valiant Labs org

@bartowski thank you for looking out, but actually, it's typical for us = it's how it saves a LORA train, that pt7 happens to only has a few parameters that don't get touched by our LORA settings, so it won't change from reupload:

"lm_head.weight": "model-00007-of-00007.safetensors",
"model.layers.31.input_layernorm.weight": "model-00007-of-00007.safetensors",
"model.layers.31.mlp.down_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.31.mlp.up_proj.weight": "model-00007-of-00007.safetensors",
"model.layers.31.post_attention_layernorm.weight": "model-00007-of-00007.safetensors",
"model.norm.weight": "model-00007-of-00007.safetensors"

no k/o/q/v :)

The upload folder did identify all 7 parts, so it did see them:

Upload 1 file(s) totalling 4.9G (adfacd1fdecad3533c359c30035a150da4a6e0744babe33836ac0fdb654fbd47)
Upload 1 file(s) totalling 4.8G (89d39802f0f44aab3aaec9fe367bbde5bdeb1f600dc741fc19855a66bf25179e)
Upload 1 file(s) totalling 5.0G (154b6296b96d19d09f03b3a2c445b70c6254c729fee49893e751a6f8bd5f4923)
Upload 1 file(s) totalling 5.0G (04e119436f75f623c3d09d0eed91cd5bdcec6c8aed307bdb0a418776a3ed37a3)
Upload 1 file(s) totalling 4.8G (aad672a5c32ccddaefef79368d8cca41096ad7c03b3d22377804b1dd0eefa803)
Upload 1 file(s) totalling 5.0G (2699bd4029ecae938e19db9fe8e756f6ed9c30214508311656496453decf4e2f)
Upload 1 file(s) totalling 2.6G (6609a37293c20b92462f30060c64401cf6deab00569ac44f66f21183915414cb)
Upload 7 file(s) totalling 17.3M (c191b597e4415146f4bb0d3df015d2bd960a03ca7ee497671c00fa16ac390ee5)

Ah okay awesome makes sense :) figured I'd double check!

On that subject, any reason to upload in place vs releasing it as 2.1?

Sign up or log in to comment