Static Embeddings using SQLiteVec
This repo contains sentence-transformers/static-retrieval-mrl-en-v1 model converted to sqlite-vec database.
This repo contains a Swift package that demonstrates how to use the SQLiteVec to compute static embeddings.
This repo contains a benchmark that compares the performance of computing embeddings using swift-embeddings and SQLiteVec.
The sqlite-vec database was created using the following command:
CREATE VIRTUAL TABLE embeddings USING vec0(embedding float[1024])
and then the embeddings were inserted using the following command:
INSERT INTO embeddings(rowid, embedding) VALUES (?, ?)
Benchmark
To run the benchmark, execute the following command:
cd EmbeddingsBenchmark
swift package --disable-sandbox benchmark
Results
CoreML benchmark (count: 10)
βββββββββββββββββββββββββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ
β Metric β p0 β p25 β p50 β p75 β p90 β p99 β p100 β Samples β
βββββββββββββββββββββββββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββ‘
β Instructions (K) * β 1712 β 1779 β 1779 β 1791 β 1792 β 1805 β 1899 β 5136 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Malloc (total) * β 1131 β 1132 β 1132 β 1132 β 1132 β 1132 β 1139 β 5136 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Memory (resident peak) (M) β 156 β 158 β 158 β 159 β 159 β 159 β 159 β 5136 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Throughput (# / s) (#) β 11121 β 9471 β 9287 β 8919 β 8607 β 7747 β 5810 β 5136 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (total CPU) (ΞΌs) * β 92 β 107 β 109 β 114 β 118 β 131 β 174 β 5136 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (wall clock) (ΞΌs) * β 90 β 106 β 108 β 112 β 116 β 129 β 172 β 5136 β
βββββββββββββββββββββββββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ
CoreML benchmark (count: 1000)
βββββββββββββββββββββββββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ
β Metric β p0 β p25 β p50 β p75 β p90 β p99 β p100 β Samples β
βββββββββββββββββββββββββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββ‘
β Instructions (K) * β 3320 β 3422 β 3467 β 3516 β 3545 β 3566 β 3640 β 2698 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Malloc (total) * β 1131 β 1132 β 1132 β 1132 β 1132 β 1132 β 1140 β 2698 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Memory (resident peak) (M) β 159 β 162 β 162 β 162 β 162 β 162 β 162 β 2698 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Throughput (# / s) (#) β 4418 β 4299 β 4231 β 4139 β 3999 β 3515 β 2093 β 2698 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (total CPU) (ΞΌs) * β 228 β 234 β 238 β 243 β 252 β 287 β 480 β 2698 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (wall clock) (ΞΌs) * β 226 β 233 β 237 β 242 β 250 β 285 β 478 β 2698 β
βββββββββββββββββββββββββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ
CoreML benchmark (count: 100000)
βββββββββββββββββββββββββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ
β Metric β p0 β p25 β p50 β p75 β p90 β p99 β p100 β Samples β
βββββββββββββββββββββββββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββ‘
β Instructions (M) * β 163 β 163 β 163 β 163 β 163 β 168 β 168 β 20 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Malloc (total) * β 1131 β 1131 β 1131 β 1132 β 1133 β 1139 β 1139 β 20 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Memory (resident peak) (M) β 566 β 567 β 567 β 568 β 568 β 568 β 568 β 20 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Throughput (# / s) (#) β 21 β 21 β 21 β 21 β 21 β 20 β 20 β 20 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (total CPU) (ms) * β 47 β 48 β 48 β 48 β 48 β 50 β 50 β 20 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (wall clock) (ms) * β 47 β 48 β 48 β 48 β 48 β 50 β 50 β 20 β
βββββββββββββββββββββββββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ
SQLite benchmark (count: 10)
βββββββββββββββββββββββββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ
β Metric β p0 β p25 β p50 β p75 β p90 β p99 β p100 β Samples β
βββββββββββββββββββββββββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββ‘
β Instructions (M) * β 103 β 117 β 122 β 128 β 132 β 136 β 140 β 144 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Malloc (total) * β 99 β 106 β 106 β 106 β 106 β 107 β 107 β 144 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Memory (resident peak) (M) β 15 β 17 β 19 β 19 β 19 β 19 β 19 β 144 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Throughput (# / s) (#) β 186 β 155 β 147 β 139 β 132 β 126 β 126 β 144 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (total CPU) (ΞΌs) * β 5382 β 6451 β 6779 β 7180 β 7582 β 7922 β 7932 β 144 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (wall clock) (ΞΌs) * β 5380 β 6443 β 6779 β 7180 β 7574 β 7918 β 7930 β 144 β
βββββββββββββββββββββββββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ
SQLite benchmark (count: 1000)
βββββββββββββββββββββββββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ€ββββββββββ
β Metric β p0 β p25 β p50 β p75 β p90 β p99 β p100 β Samples β
βββββββββββββββββββββββββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββͺββββββββββ‘
β Instructions (M) * β 1517 β 1572 β 1599 β 1611 β 1618 β 1618 β 1618 β 9 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Malloc (total) * β 6330 β 6339 β 6367 β 6379 β 6414 β 6414 β 6414 β 9 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Memory (resident peak) (M) β 22 β 23 β 23 β 23 β 23 β 23 β 23 β 9 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Throughput (# / s) (#) β 9 β 8 β 8 β 8 β 8 β 8 β 8 β 9 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (total CPU) (ms) * β 114 β 118 β 120 β 121 β 121 β 121 β 121 β 9 β
βββββββββββββββββββββββββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββΌββββββββββ€
β Time (wall clock) (ms) * β 114 β 118 β 120 β 121 β 121 β 121 β 121 β 9 β
βββββββββββββββββββββββββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ§ββββββββββ
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.