File size: 1,216 Bytes
29f7dbc ad7ebb6 29f7dbc ad7ebb6 c09b9cb ad7ebb6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
---
language:
- en
license: mit
---
This model was trained with [Neural-Cherche](https://github.com/raphaelsty/neural-cherche). You can find details on how to fine-tune it in the [Neural-Cherche](https://github.com/raphaelsty/neural-cherche) repository.
```sh
pip install neural-cherche
```
## Retriever
```python
from neural_cherche import models, retrieve
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
batch_size = 32
documents = [
{"id": 0, "document": "Food"},
{"id": 1, "document": "Sports"},
{"id": 2, "document": "Cinema"},
]
queries = ["Food", "Sports", "Cinema"]
model = models.SparseEmbed(
model_name_or_path="raphaelsty/neural-cherche-sparse-embed",
device=device,
)
retriever = retrieve.SparseEmbed(
key="id",
on=["document"],
model=model,
)
documents_embeddings = retriever.encode_documents(
documents=documents,
batch_size=batch_size,
)
retriever = retriever.add(
documents_embeddings=documents_embeddings,
)
queries_embeddings = retriever.encode_queries(
queries=queries,
batch_size=batch_size,
)
scores = retriever(
queries_embeddings=queries_embeddings,
batch_size=batch_size,
k=100,
)
scores
```
|