wow

#1
by MWilinski - opened

😃: what are huggingface safetensors?
🤖: Safetensors is a new simple format for storing tensors safely (as opposed to pickle) and that is still fast (zero-copy). Safetensors is really fast 🚀.


### Usage

```python
import torch
from safetensors import safe_open
from safetensors.torch import save_file

tensors = {}
with safe_open("model.saferos((2, 3))
}
save_file(tensors, "model.safetensors")

Format

Let's say you have safetensors file named model.safetensors, then model.safetensors will have the following internal format:

Featured Projects

Safetensors is being used widely at leading AI enterprises, such as Hugging Face, EleutherAI, and StabilityAI. Here is a non-exhaustive list of projects that are using safetensors:

safetensors

Safetensors

This repository implements a new simple format for storing tensors
safely (as opposed to pickle) and that is still fast (zero-copy).

Installation

Pip

You can install safetensors via the pip manager:

pip install safetensors

From source

For the sources, you need Rust

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Make sure it's up to date and using stable channel
rustup update
git clone https://github.com/huggingface/safetensors
cd safetensors/bindings/python
pip install setuptools_rust
pip install -e.

Getting started

import torch
from safetensors import safe_open
from safetensors.torch import save_file

te
### Input:
what are huggingface safetensors?
### Response:
Safetensors is a new simple format for storing tensors safely (as opposed to pickle) and that is still fast (zero-copy). Safetensors is really [fast 🚀](./speed).

Usage

import torch
from safetensors import safe_open
from safetensors.torch import save_file

tensors = {}
with safe_open("model.saferos((2, 3))
}
save_file(tensors, "model.safetensors")

Format

Let's say you have safetensors file named model.safetensors, then model.safetensors will have the following internal format:

Featured Projects

Safetensors is being used widely at leading AI enterprises, such as Hugging Face, EleutherAI, and StabilityAI. Here is a non-exhaustive list of projects that are using safetensors:

safetensors

Safetensors

This repository implements a new simple format for storing tensors
safely (as opposed to pickle) and that is still fast (zero-copy).

Installation

Pip

You can install safetensors via the pip manager:

pip install safetensors

From source

For the sources, you need Rust

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Make sure it's up to date and using stable channel
rustup update
git clone https://github.com/huggingface/safetensors
cd safetensors/bindings/python
pip install setuptools_rust
pip install -e.

Getting started

import torch
from safetensors import safe_open
from safetensors.torch import save_file

te
### Input:
what are huggingface safetensors?
### Response:
Safetensors is a new simple format for storing tensors safely (as opposed to pickle) and that is still fast (zero-copy). Safetensors is really [fast 🚀](./speed).

Usage

import torch
from safetensors import safe_open
from safetensors.torch import save_file

tensors = {}
with safe_open("model.saferos((2, 3))
}
save_file(tensors, "model.safetensors")

Format

Let's say you have safetensors file named model.safetensors, then model.safetensors will have the following internal format:

Featured Projects

Safetensors is being used widely at leading AI enterprises, such as Hugging Face, EleutherAI, and StabilityAI. Here is a non-exhaustive list of projects that are using safetensors:

safetensors

Safetensors

This repository implements a new simple format for storing tensors
safely (as opposed to pickle) and that is still fast (zero-copy).

Installation

Pip

You can install safetensors via the pip manager:

pip install safetensors

From source

For the sources, you need Rust

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Make sure it's up to date and using stable channel
rustup update
git clone https://github.com/huggingface/safetensors
cd safetensors/bindings/python
pip install setuptools_rust
pip install -e.

Getting started

import torch
from safetensors import safe

Sources:
 [1] https://github.com/huggingface/safetensors/blob/main/README.md
 [2] https://github.com/huggingface/safetensors/blob/main/docs/source/index.mdx

Sign up or log in to comment