Hugging Face
Storage Buckets Storage Buckets

Next-gen large-scale storage for AI

Xet uses content-defined chunking to break files into byte-level chunks and deduplicates across your entire bucket. When you retrain a model and only 5% of weights change, only that 5% is re-uploaded.

• Raw + processed dataset: stored once, billed once*

• 4x less data per upload, verified with real-world workloads

*Requires Enterprise or Enterprise Plus plan

Assemble training data at any scale

Pour raw data from every source into a single bucket: crawls, annotations, synthetic outputs, partner datasets. No git overhead, no commit queues, no file-count limits. When training begins, your data is already there, streamed to GPUs via the included CDN.

• Immediate availability on upload, no queued commits

• Batch API processes thousands of files in a single call

• Raw + processed datasets with dedup = no double billing*

*Requires Enterprise or Enterprise Plus plan

Give your coding agents persistent storage

Coding agents run in ephemeral environments, but their outputs shouldn't vanish. Checkpoints, benchmark results, generated datasets: one hf sync command in your agent's bash tool is all it takes.

• Pre-warmed CDN and no git overhead for fast reads and writes

• Persist artifacts across ephemeral CI runs and terminal sessions

• Install the official HF CLI skill and your agent knows every command