|
--- |
|
base_model: actualdata/bilingual-embedding-large |
|
datasets: [] |
|
language: |
|
- en |
|
library_name: sentence-transformers |
|
license: apache-2.0 |
|
metrics: |
|
- cosine_accuracy@1 |
|
- cosine_accuracy@3 |
|
- cosine_accuracy@5 |
|
- cosine_accuracy@10 |
|
- cosine_precision@1 |
|
- cosine_precision@3 |
|
- cosine_precision@5 |
|
- cosine_precision@10 |
|
- cosine_recall@1 |
|
- cosine_recall@3 |
|
- cosine_recall@5 |
|
- cosine_recall@10 |
|
- cosine_ndcg@10 |
|
- cosine_mrr@10 |
|
- cosine_map@100 |
|
pipeline_tag: sentence-similarity |
|
tags: |
|
- sentence-transformers |
|
- sentence-similarity |
|
- feature-extraction |
|
- generated_from_trainer |
|
- dataset_size:4885 |
|
- loss:MatryoshkaLoss |
|
- loss:MultipleNegativesRankingLoss |
|
widget: |
|
- source_sentence: ' Le CO2, le CH4, le N2O, le SF6, le NF3 ainsi que les groupes |
|
de gaz HFC et PFC.' |
|
sentences: |
|
- ' Qui a initié l''élaboration du guide sectoriel de réalisation d''un bilan des |
|
émissions de gaz à effet de serre pour la filière cosmétique ?' |
|
- ' Quel est l''objectif premier du Guide sectoriel de réalisation d''un bilan des |
|
émissions de gaz à effet de serre pour la filière des sites de loisirs et culturels |
|
?' |
|
- ' Quel est le gaz contribuant à l''augmentation de l''effet de serre qui doit |
|
être pris en compte dans la réalisation des bilans ?' |
|
- source_sentence: ' Il est conseillé d''implémenter d''abord les leviers déjà matures |
|
et « sans regret » (efficacité énergétique, efficacité matières, décarbonation |
|
du mix énergétique) avant d''envisager des technologies moins matures.' |
|
sentences: |
|
- ' Quel est le recommandé ordre d''implémentation des leviers de décarbonation |
|
?' |
|
- ' Quels sont les types de connexions utilisés pour relier un utilisateur à une |
|
ressource distante dans un réseau de communication ?' |
|
- ' Comment peut-on utiliser le Bilan Carbone pour tenir compte de processus de |
|
valorisation mis en œuvre par les entreprises du secteur agricole et agro-alimentaire |
|
?' |
|
- source_sentence: ' Les échanges ont permis de décrire des exemples par poste d''émissions.' |
|
sentences: |
|
- ' Quel était l''objectif des échanges sur les bonnes pratiques utilisées dans |
|
le secteur ?' |
|
- Existe-t-il une méthode rigoureuse pour déterminer l'incertitude de ces facteurs |
|
d'émissions monétaires? |
|
- ' Quels sont les modes de transport pris en compte dans cette fiche ?' |
|
- source_sentence: ' La variation du périmètre organisationnel par la vente d''une |
|
usine, la variation du périmètre opérationnel par l''achat d''une nouvelle ligne |
|
de production, le changement de valeur de facteurs d''émission, le changement |
|
du mix des produits des usines et la dégradation des outils de production.' |
|
sentences: |
|
- ' Quel type de repas a un total de quantité (g) de 83229,6 ? ' |
|
- Quel est l'objectif principal de la collecte des données pour la réalisation d'un |
|
bilan GES ? |
|
- ' Quels sont les facteurs qui ont influencé l''évolution des émissions de GES |
|
de l''entreprise ?' |
|
- source_sentence: ' Le PCS intègre l''énergie libérée par la condensation de l''eau |
|
après la combustion, tandis que le PCI ne l''intègre pas.' |
|
sentences: |
|
- ' La proportion d''énergie utilisée dans l''eau chaude sanitaire pour les résidences |
|
principales (métropole uniquement) est-elle supérieure à 1 % ?' |
|
- ' Qu''est-ce qui distingue le Pouvoir Calorifique Supérieur (PCS) du Pouvoir Calorifique |
|
Inférieur (PCI) ?' |
|
- ' Quelle méthode de mesure directe par suivi de la consommation des véhicules |
|
de transport sera privilégiée si le matériel de transport est contrôlé ?' |
|
model-index: |
|
- name: test qwen2 Matryoshka |
|
results: |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 1024 |
|
type: dim_1024 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.31675874769797424 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.425414364640884 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.47697974217311234 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.5561694290976059 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.31675874769797424 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.141804788213628 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.09539594843462246 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.05561694290976059 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.31675874769797424 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.425414364640884 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.47697974217311234 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.5561694290976059 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.42756869844177203 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.38761729369464176 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.399364505533715 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 896 |
|
type: dim_896 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.32228360957642727 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.42357274401473294 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.4732965009208103 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.5488029465930019 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.32228360957642727 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.14119091467157763 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.09465930018416206 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.05488029465930018 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.32228360957642727 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.42357274401473294 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.4732965009208103 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.5488029465930019 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.4272124343988002 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.3893734105060072 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.40183454050045436 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 512 |
|
type: dim_512 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.3314917127071823 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.42357274401473294 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.47513812154696133 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.5488029465930019 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.3314917127071823 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.14119091467157763 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.09502762430939225 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.05488029465930018 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.3314917127071823 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.42357274401473294 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.47513812154696133 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.5488029465930019 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.43088591845526986 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.39430705369931895 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.4065191633235482 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 256 |
|
type: dim_256 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.30755064456721914 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.4125230202578269 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.4677716390423573 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.5395948434622467 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.30755064456721914 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.1375076734192756 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.09355432780847145 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.053959484346224676 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.30755064456721914 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.4125230202578269 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.4677716390423573 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.5395948434622467 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.41562425407928066 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.3769351632611302 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.3895577962122803 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 128 |
|
type: dim_128 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.2965009208103131 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.40515653775322286 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.44751381215469616 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.5395948434622467 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.2965009208103131 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.13505217925107427 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.08950276243093921 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.053959484346224676 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.2965009208103131 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.40515653775322286 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.44751381215469616 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.5395948434622467 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.40786326501955955 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.367228653278377 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.3789438619494699 |
|
name: Cosine Map@100 |
|
--- |
|
|
|
# test qwen2 Matryoshka |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [actualdata/bilingual-embedding-large](https://huggingface.co/actualdata/bilingual-embedding-large). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** Sentence Transformer |
|
- **Base model:** [actualdata/bilingual-embedding-large](https://huggingface.co/actualdata/bilingual-embedding-large) <!-- at revision b595d8ed97b05e847230c8bd2432ea248c2afe2d --> |
|
- **Maximum Sequence Length:** 512 tokens |
|
- **Output Dimensionality:** 1024 tokens |
|
- **Similarity Function:** Cosine Similarity |
|
<!-- - **Training Dataset:** Unknown --> |
|
- **Language:** en |
|
- **License:** apache-2.0 |
|
|
|
### Model Sources |
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) |
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) |
|
|
|
### Full Model Architecture |
|
|
|
``` |
|
SentenceTransformer( |
|
(0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BilingualModel |
|
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
(2): Normalize() |
|
) |
|
``` |
|
|
|
## Usage |
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
First install the Sentence Transformers library: |
|
|
|
```bash |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can load this model and run inference. |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
|
|
# Download from the 🤗 Hub |
|
model = SentenceTransformer("sylvain471/bl_ademe_large") |
|
# Run inference |
|
sentences = [ |
|
" Le PCS intègre l'énergie libérée par la condensation de l'eau après la combustion, tandis que le PCI ne l'intègre pas.", |
|
" Qu'est-ce qui distingue le Pouvoir Calorifique Supérieur (PCS) du Pouvoir Calorifique Inférieur (PCI) ?", |
|
" La proportion d'énergie utilisée dans l'eau chaude sanitaire pour les résidences principales (métropole uniquement) est-elle supérieure à 1 % ?", |
|
] |
|
embeddings = model.encode(sentences) |
|
print(embeddings.shape) |
|
# [3, 1024] |
|
|
|
# Get the similarity scores for the embeddings |
|
similarities = model.similarity(embeddings, embeddings) |
|
print(similarities.shape) |
|
# [3, 3] |
|
``` |
|
|
|
<!-- |
|
### Direct Usage (Transformers) |
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Downstream Usage (Sentence Transformers) |
|
|
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_1024` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.3168 | |
|
| cosine_accuracy@3 | 0.4254 | |
|
| cosine_accuracy@5 | 0.477 | |
|
| cosine_accuracy@10 | 0.5562 | |
|
| cosine_precision@1 | 0.3168 | |
|
| cosine_precision@3 | 0.1418 | |
|
| cosine_precision@5 | 0.0954 | |
|
| cosine_precision@10 | 0.0556 | |
|
| cosine_recall@1 | 0.3168 | |
|
| cosine_recall@3 | 0.4254 | |
|
| cosine_recall@5 | 0.477 | |
|
| cosine_recall@10 | 0.5562 | |
|
| cosine_ndcg@10 | 0.4276 | |
|
| cosine_mrr@10 | 0.3876 | |
|
| **cosine_map@100** | **0.3994** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_896` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.3223 | |
|
| cosine_accuracy@3 | 0.4236 | |
|
| cosine_accuracy@5 | 0.4733 | |
|
| cosine_accuracy@10 | 0.5488 | |
|
| cosine_precision@1 | 0.3223 | |
|
| cosine_precision@3 | 0.1412 | |
|
| cosine_precision@5 | 0.0947 | |
|
| cosine_precision@10 | 0.0549 | |
|
| cosine_recall@1 | 0.3223 | |
|
| cosine_recall@3 | 0.4236 | |
|
| cosine_recall@5 | 0.4733 | |
|
| cosine_recall@10 | 0.5488 | |
|
| cosine_ndcg@10 | 0.4272 | |
|
| cosine_mrr@10 | 0.3894 | |
|
| **cosine_map@100** | **0.4018** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_512` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.3315 | |
|
| cosine_accuracy@3 | 0.4236 | |
|
| cosine_accuracy@5 | 0.4751 | |
|
| cosine_accuracy@10 | 0.5488 | |
|
| cosine_precision@1 | 0.3315 | |
|
| cosine_precision@3 | 0.1412 | |
|
| cosine_precision@5 | 0.095 | |
|
| cosine_precision@10 | 0.0549 | |
|
| cosine_recall@1 | 0.3315 | |
|
| cosine_recall@3 | 0.4236 | |
|
| cosine_recall@5 | 0.4751 | |
|
| cosine_recall@10 | 0.5488 | |
|
| cosine_ndcg@10 | 0.4309 | |
|
| cosine_mrr@10 | 0.3943 | |
|
| **cosine_map@100** | **0.4065** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_256` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.3076 | |
|
| cosine_accuracy@3 | 0.4125 | |
|
| cosine_accuracy@5 | 0.4678 | |
|
| cosine_accuracy@10 | 0.5396 | |
|
| cosine_precision@1 | 0.3076 | |
|
| cosine_precision@3 | 0.1375 | |
|
| cosine_precision@5 | 0.0936 | |
|
| cosine_precision@10 | 0.054 | |
|
| cosine_recall@1 | 0.3076 | |
|
| cosine_recall@3 | 0.4125 | |
|
| cosine_recall@5 | 0.4678 | |
|
| cosine_recall@10 | 0.5396 | |
|
| cosine_ndcg@10 | 0.4156 | |
|
| cosine_mrr@10 | 0.3769 | |
|
| **cosine_map@100** | **0.3896** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_128` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.2965 | |
|
| cosine_accuracy@3 | 0.4052 | |
|
| cosine_accuracy@5 | 0.4475 | |
|
| cosine_accuracy@10 | 0.5396 | |
|
| cosine_precision@1 | 0.2965 | |
|
| cosine_precision@3 | 0.1351 | |
|
| cosine_precision@5 | 0.0895 | |
|
| cosine_precision@10 | 0.054 | |
|
| cosine_recall@1 | 0.2965 | |
|
| cosine_recall@3 | 0.4052 | |
|
| cosine_recall@5 | 0.4475 | |
|
| cosine_recall@10 | 0.5396 | |
|
| cosine_ndcg@10 | 0.4079 | |
|
| cosine_mrr@10 | 0.3672 | |
|
| **cosine_map@100** | **0.3789** | |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Dataset |
|
|
|
#### Unnamed Dataset |
|
|
|
|
|
* Size: 4,885 training samples |
|
* Columns: <code>positive</code> and <code>anchor</code> |
|
* Approximate statistics based on the first 1000 samples: |
|
| | positive | anchor | |
|
|:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| |
|
| type | string | string | |
|
| details | <ul><li>min: 3 tokens</li><li>mean: 32.82 tokens</li><li>max: 185 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 26.77 tokens</li><li>max: 71 tokens</li></ul> | |
|
* Samples: |
|
| positive | anchor | |
|
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------| |
|
| <code> Lorsque le traitement spécifique par catégorie de déchets produits par la Personne Morale est inconnu, le taux moyen local ou sectoriel de traitement en fin de vie (incinération, mise en décharge, recyclage, compostage, etc.) est utilisé. Le transport est également un paramètre à intégrer au calcul.</code> | <code> Quels sont les paramètres clés par type de traitement à prendre en compte pour réaliser un bilan d'émissions de gaz à effet de serre ?</code> | |
|
| <code> Une analyse de cycle de vie fournit un moyen efficace et systémique pour évaluer les impacts environnementaux d’un produit, d’un service, d’une entreprise ou d’un procédé.</code> | <code> Qu'est-ce qu'une évaluation de cycle de vie (ACV) ?</code> | |
|
| <code> 1 469,2 t CO2e.</code> | <code> Quel est le total des émissions annuelles de l'entreprise GAMMA ?</code> | |
|
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: |
|
```json |
|
{ |
|
"loss": "MultipleNegativesRankingLoss", |
|
"matryoshka_dims": [ |
|
1024, |
|
896, |
|
512, |
|
256, |
|
128 |
|
], |
|
"matryoshka_weights": [ |
|
1, |
|
1, |
|
1, |
|
1, |
|
1 |
|
], |
|
"n_dims_per_step": -1 |
|
} |
|
``` |
|
|
|
### Training Hyperparameters |
|
#### Non-Default Hyperparameters |
|
|
|
- `eval_strategy`: epoch |
|
- `per_device_train_batch_size`: 16 |
|
- `gradient_accumulation_steps`: 8 |
|
- `learning_rate`: 2e-05 |
|
- `num_train_epochs`: 20 |
|
- `lr_scheduler_type`: cosine |
|
- `warmup_ratio`: 0.1 |
|
- `bf16`: True |
|
- `tf32`: True |
|
- `load_best_model_at_end`: True |
|
- `optim`: adamw_torch_fused |
|
- `batch_sampler`: no_duplicates |
|
|
|
#### All Hyperparameters |
|
<details><summary>Click to expand</summary> |
|
|
|
- `overwrite_output_dir`: False |
|
- `do_predict`: False |
|
- `eval_strategy`: epoch |
|
- `prediction_loss_only`: True |
|
- `per_device_train_batch_size`: 16 |
|
- `per_device_eval_batch_size`: 8 |
|
- `per_gpu_train_batch_size`: None |
|
- `per_gpu_eval_batch_size`: None |
|
- `gradient_accumulation_steps`: 8 |
|
- `eval_accumulation_steps`: None |
|
- `torch_empty_cache_steps`: None |
|
- `learning_rate`: 2e-05 |
|
- `weight_decay`: 0.0 |
|
- `adam_beta1`: 0.9 |
|
- `adam_beta2`: 0.999 |
|
- `adam_epsilon`: 1e-08 |
|
- `max_grad_norm`: 1.0 |
|
- `num_train_epochs`: 20 |
|
- `max_steps`: -1 |
|
- `lr_scheduler_type`: cosine |
|
- `lr_scheduler_kwargs`: {} |
|
- `warmup_ratio`: 0.1 |
|
- `warmup_steps`: 0 |
|
- `log_level`: passive |
|
- `log_level_replica`: warning |
|
- `log_on_each_node`: True |
|
- `logging_nan_inf_filter`: True |
|
- `save_safetensors`: True |
|
- `save_on_each_node`: False |
|
- `save_only_model`: False |
|
- `restore_callback_states_from_checkpoint`: False |
|
- `no_cuda`: False |
|
- `use_cpu`: False |
|
- `use_mps_device`: False |
|
- `seed`: 42 |
|
- `data_seed`: None |
|
- `jit_mode_eval`: False |
|
- `use_ipex`: False |
|
- `bf16`: True |
|
- `fp16`: False |
|
- `fp16_opt_level`: O1 |
|
- `half_precision_backend`: auto |
|
- `bf16_full_eval`: False |
|
- `fp16_full_eval`: False |
|
- `tf32`: True |
|
- `local_rank`: 0 |
|
- `ddp_backend`: None |
|
- `tpu_num_cores`: None |
|
- `tpu_metrics_debug`: False |
|
- `debug`: [] |
|
- `dataloader_drop_last`: False |
|
- `dataloader_num_workers`: 0 |
|
- `dataloader_prefetch_factor`: None |
|
- `past_index`: -1 |
|
- `disable_tqdm`: False |
|
- `remove_unused_columns`: True |
|
- `label_names`: None |
|
- `load_best_model_at_end`: True |
|
- `ignore_data_skip`: False |
|
- `fsdp`: [] |
|
- `fsdp_min_num_params`: 0 |
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
- `deepspeed`: None |
|
- `label_smoothing_factor`: 0.0 |
|
- `optim`: adamw_torch_fused |
|
- `optim_args`: None |
|
- `adafactor`: False |
|
- `group_by_length`: False |
|
- `length_column_name`: length |
|
- `ddp_find_unused_parameters`: None |
|
- `ddp_bucket_cap_mb`: None |
|
- `ddp_broadcast_buffers`: False |
|
- `dataloader_pin_memory`: True |
|
- `dataloader_persistent_workers`: False |
|
- `skip_memory_metrics`: True |
|
- `use_legacy_prediction_loop`: False |
|
- `push_to_hub`: False |
|
- `resume_from_checkpoint`: None |
|
- `hub_model_id`: None |
|
- `hub_strategy`: every_save |
|
- `hub_private_repo`: False |
|
- `hub_always_push`: False |
|
- `gradient_checkpointing`: False |
|
- `gradient_checkpointing_kwargs`: None |
|
- `include_inputs_for_metrics`: False |
|
- `eval_do_concat_batches`: True |
|
- `fp16_backend`: auto |
|
- `push_to_hub_model_id`: None |
|
- `push_to_hub_organization`: None |
|
- `mp_parameters`: |
|
- `auto_find_batch_size`: False |
|
- `full_determinism`: False |
|
- `torchdynamo`: None |
|
- `ray_scope`: last |
|
- `ddp_timeout`: 1800 |
|
- `torch_compile`: False |
|
- `torch_compile_backend`: None |
|
- `torch_compile_mode`: None |
|
- `dispatch_batches`: None |
|
- `split_batches`: None |
|
- `include_tokens_per_second`: False |
|
- `include_num_input_tokens_seen`: False |
|
- `neftune_noise_alpha`: None |
|
- `optim_target_modules`: None |
|
- `batch_eval_metrics`: False |
|
- `eval_on_start`: False |
|
- `eval_use_gather_object`: False |
|
- `batch_sampler`: no_duplicates |
|
- `multi_dataset_batch_sampler`: proportional |
|
|
|
</details> |
|
|
|
### Training Logs |
|
| Epoch | Step | Training Loss | dim_1024_cosine_map@100 | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_896_cosine_map@100 | |
|
|:-----------:|:-------:|:-------------:|:-----------------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:| |
|
| 0.2614 | 10 | 5.4141 | - | - | - | - | - | |
|
| 0.5229 | 20 | 4.2823 | - | - | - | - | - | |
|
| 0.7843 | 30 | 3.0162 | - | - | - | - | - | |
|
| 0.9935 | 38 | - | 0.3636 | 0.3170 | 0.3407 | 0.3566 | 0.3668 | |
|
| 1.0458 | 40 | 2.5846 | - | - | - | - | - | |
|
| 1.3072 | 50 | 2.2069 | - | - | - | - | - | |
|
| 1.5686 | 60 | 1.7585 | - | - | - | - | - | |
|
| 1.8301 | 70 | 1.3099 | - | - | - | - | - | |
|
| 1.9869 | 76 | - | 0.3979 | 0.3353 | 0.3726 | 0.3895 | 0.3983 | |
|
| 2.0915 | 80 | 1.1449 | - | - | - | - | - | |
|
| 2.3529 | 90 | 1.0137 | - | - | - | - | - | |
|
| 2.6144 | 100 | 0.6402 | - | - | - | - | - | |
|
| 2.8758 | 110 | 0.4931 | - | - | - | - | - | |
|
| 2.9804 | 114 | - | 0.4026 | 0.3568 | 0.3808 | 0.3882 | 0.3992 | |
|
| 3.1373 | 120 | 0.4662 | - | - | - | - | - | |
|
| 3.3987 | 130 | 0.3782 | - | - | - | - | - | |
|
| 3.6601 | 140 | 0.2696 | - | - | - | - | - | |
|
| 3.9216 | 150 | 0.2478 | - | - | - | - | - | |
|
| 4.0 | 153 | - | 0.3805 | 0.3460 | 0.3613 | 0.3680 | 0.3850 | |
|
| 4.1830 | 160 | 0.2655 | - | - | - | - | - | |
|
| 4.4444 | 170 | 0.1952 | - | - | - | - | - | |
|
| 4.7059 | 180 | 0.1494 | - | - | - | - | - | |
|
| 4.9673 | 190 | 0.1482 | - | - | - | - | - | |
|
| 4.9935 | 191 | - | 0.3806 | 0.3619 | 0.3702 | 0.3799 | 0.3814 | |
|
| 5.2288 | 200 | 0.161 | - | - | - | - | - | |
|
| 5.4902 | 210 | 0.1282 | - | - | - | - | - | |
|
| 5.7516 | 220 | 0.0888 | - | - | - | - | - | |
|
| 5.9869 | 229 | - | 0.3936 | 0.3685 | 0.3758 | 0.3870 | 0.3916 | |
|
| 6.0131 | 230 | 0.1042 | - | - | - | - | - | |
|
| 6.2745 | 240 | 0.126 | - | - | - | - | - | |
|
| 6.5359 | 250 | 0.103 | - | - | - | - | - | |
|
| 6.7974 | 260 | 0.0467 | - | - | - | - | - | |
|
| 6.9804 | 267 | - | 0.4022 | 0.3689 | 0.3897 | 0.3950 | 0.4022 | |
|
| 7.0588 | 270 | 0.0581 | - | - | - | - | - | |
|
| 7.3203 | 280 | 0.0728 | - | - | - | - | - | |
|
| 7.5817 | 290 | 0.064 | - | - | - | - | - | |
|
| 7.8431 | 300 | 0.0271 | - | - | - | - | - | |
|
| 8.0 | 306 | - | 0.4010 | 0.3756 | 0.3872 | 0.3988 | 0.4021 | |
|
| 8.1046 | 310 | 0.0452 | - | - | - | - | - | |
|
| 8.3660 | 320 | 0.0613 | - | - | - | - | - | |
|
| 8.6275 | 330 | 0.0294 | - | - | - | - | - | |
|
| 8.8889 | 340 | 0.0396 | - | - | - | - | - | |
|
| 8.9935 | 344 | - | 0.3914 | 0.3722 | 0.3801 | 0.3916 | 0.3939 | |
|
| 9.1503 | 350 | 0.024 | - | - | - | - | - | |
|
| 9.4118 | 360 | 0.0253 | - | - | - | - | - | |
|
| 9.6732 | 370 | 0.017 | - | - | - | - | - | |
|
| 9.9346 | 380 | 0.0163 | - | - | - | - | - | |
|
| 9.9869 | 382 | - | 0.3901 | 0.3660 | 0.3796 | 0.3892 | 0.3904 | |
|
| 10.1961 | 390 | 0.0191 | - | - | - | - | - | |
|
| 10.4575 | 400 | 0.017 | - | - | - | - | - | |
|
| 10.7190 | 410 | 0.0108 | - | - | - | - | - | |
|
| **10.9804** | **420** | **0.0118** | **0.3994** | **0.3789** | **0.3896** | **0.4065** | **0.4018** | |
|
| 11.2418 | 430 | 0.0111 | - | - | - | - | - | |
|
| 11.5033 | 440 | 0.011 | - | - | - | - | - | |
|
| 11.7647 | 450 | 0.0052 | - | - | - | - | - | |
|
| 12.0 | 459 | - | 0.4030 | 0.3772 | 0.3986 | 0.4034 | 0.3999 | |
|
| 12.0261 | 460 | 0.0144 | - | - | - | - | - | |
|
| 12.2876 | 470 | 0.0068 | - | - | - | - | - | |
|
| 12.5490 | 480 | 0.0061 | - | - | - | - | - | |
|
| 12.8105 | 490 | 0.0039 | - | - | - | - | - | |
|
| 12.9935 | 497 | - | 0.4022 | 0.3733 | 0.3869 | 0.3995 | 0.3983 | |
|
| 13.0719 | 500 | 0.0074 | - | - | - | - | - | |
|
| 13.3333 | 510 | 0.005 | - | - | - | - | - | |
|
| 13.5948 | 520 | 0.0045 | - | - | - | - | - | |
|
| 13.8562 | 530 | 0.0035 | - | - | - | - | - | |
|
| 13.9869 | 535 | - | 0.4027 | 0.3779 | 0.3891 | 0.4015 | 0.3999 | |
|
| 14.1176 | 540 | 0.0047 | - | - | - | - | - | |
|
| 14.3791 | 550 | 0.0043 | - | - | - | - | - | |
|
| 14.6405 | 560 | 0.0038 | - | - | - | - | - | |
|
| 14.9020 | 570 | 0.0034 | - | - | - | - | - | |
|
| 14.9804 | 573 | - | 0.3954 | 0.3734 | 0.3875 | 0.3982 | 0.3962 | |
|
| 15.1634 | 580 | 0.0037 | - | - | - | - | - | |
|
| 15.4248 | 590 | 0.0039 | - | - | - | - | - | |
|
| 15.6863 | 600 | 0.0034 | - | - | - | - | - | |
|
| 15.9477 | 610 | 0.0033 | - | - | - | - | - | |
|
| 16.0 | 612 | - | 0.3966 | 0.3720 | 0.3852 | 0.3948 | 0.3936 | |
|
| 16.2092 | 620 | 0.0038 | - | - | - | - | - | |
|
| 16.4706 | 630 | 0.0034 | - | - | - | - | - | |
|
| 16.7320 | 640 | 0.0029 | - | - | - | - | - | |
|
| 16.9935 | 650 | 0.0033 | 0.3968 | 0.3723 | 0.3844 | 0.3977 | 0.3966 | |
|
| 17.2549 | 660 | 0.0034 | - | - | - | - | - | |
|
| 17.5163 | 670 | 0.0033 | - | - | - | - | - | |
|
| 17.7778 | 680 | 0.0028 | - | - | - | - | - | |
|
| 17.9869 | 688 | - | 0.3965 | 0.3695 | 0.3861 | 0.3960 | 0.3969 | |
|
| 18.0392 | 690 | 0.0033 | - | - | - | - | - | |
|
| 18.3007 | 700 | 0.0033 | - | - | - | - | - | |
|
| 18.5621 | 710 | 0.0036 | - | - | - | - | - | |
|
| 18.8235 | 720 | 0.0026 | - | - | - | - | - | |
|
| 18.9804 | 726 | - | 0.3962 | 0.3701 | 0.3819 | 0.3951 | 0.3964 | |
|
| 19.0850 | 730 | 0.003 | - | - | - | - | - | |
|
| 19.3464 | 740 | 0.0036 | - | - | - | - | - | |
|
| 19.6078 | 750 | 0.0033 | - | - | - | - | - | |
|
| 19.8693 | 760 | 0.0031 | 0.3994 | 0.3789 | 0.3896 | 0.4065 | 0.4018 | |
|
|
|
* The bold row denotes the saved checkpoint. |
|
|
|
### Framework Versions |
|
- Python: 3.10.12 |
|
- Sentence Transformers: 3.0.1 |
|
- Transformers: 4.44.2 |
|
- PyTorch: 2.4.1+cu121 |
|
- Accelerate: 0.34.2 |
|
- Datasets: 2.21.0 |
|
- Tokenizers: 0.19.1 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
|
|
#### Sentence Transformers |
|
```bibtex |
|
@inproceedings{reimers-2019-sentence-bert, |
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
month = "11", |
|
year = "2019", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://arxiv.org/abs/1908.10084", |
|
} |
|
``` |
|
|
|
#### MatryoshkaLoss |
|
```bibtex |
|
@misc{kusupati2024matryoshka, |
|
title={Matryoshka Representation Learning}, |
|
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, |
|
year={2024}, |
|
eprint={2205.13147}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.LG} |
|
} |
|
``` |
|
|
|
#### MultipleNegativesRankingLoss |
|
```bibtex |
|
@misc{henderson2017efficient, |
|
title={Efficient Natural Language Response Suggestion for Smart Reply}, |
|
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, |
|
year={2017}, |
|
eprint={1705.00652}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |
