Hugging Face's logo

haophancs
/
bge-m3-financial-matryoshka

Sentence Similarity
sentence-transformers
ONNX
Safetensors
English
xlm-roberta
feature-extraction
Generated from Trainer
dataset_size:6300
loss:MatryoshkaLoss
loss:MultipleNegativesRankingLoss
Eval Results
text-embeddings-inference
Inference Endpoints
Model card Files Community
bge-m3-financial-matryoshka / README.md
haophancs's picture
haophancs
Add new SentenceTransformer model.
adf982e verified
preview code
|
raw
history blame
70.7 kB
---
language:
- en
license: apache-2.0
library_name: sentence-transformers
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:6300
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-m3
datasets: []
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
widget:
- source_sentence: The consolidated financial statements and accompanying notes listed
in Part IV, Item 15(a)(1) of this Annual Report on Form 10-K.
sentences:
- How much total space does an average The Home Depot store encompass including
its garden area?
- What section of the Annual Report on Form 10-K contains the consolidated financial
statements and accompanying notes?
- What types of competitive factors does Garmin believe are important in its markets?
- source_sentence: Item 3. Legal Proceedings, which covers litigation and regulatory
matters, refers to Note 12 – Commitments and Contingencies for more detailed information
within the Consolidated Financial Statements.
sentences:
- What pages contain the Financial Statements and Supplementary Data in IBM’s 2023
Annual Report to Stockholders?
- In which note can further details on Legal Proceedings be found within the Consolidated
Financial Statements?
- What is the title of Item 8 in the document?
- source_sentence: Net Revenues for the Entertainment segment were $659.3 million
in 2023.
sentences:
- What were the net revenues for the Entertainment segment in 2023?
- How much net cash was provided by operating activities in 2023?
- What was the net income reported for the fiscal year ending in August 2023?
- source_sentence: 'The capital allocation program focuses on three objectives: (1)
grow our business at an average target ROIC-adjusted rate of 20% or greater; (2)
maintain a strong investment-grade balance sheet, including a target average automotive
cash balance of $18.0 billion; and (3) after the first two objectives are met,
return available cash to shareholders.'
sentences:
- Why is ICE Mortgage Technology subject to the examination by the Federal Financial
Institutions Examination Council (FFIEC) and its member agencies?
- What type of regulations do U.S. automobiles need to comply with under the National
Highway Traffic Safety Administration?
- What are the three objectives of the capital allocation program referenced?
- source_sentence: As of January 28, 2024 the net carrying value of our inventories
was $1.3 billion, which included provisions for obsolete and damaged inventory
of $139.7 million.
sentences:
- What is the status of the company's inventory as of January 28, 2024, in terms
of its valuation and provisions for obsolescence?
- What is the relationship between the ESG goals and the long-term growth strategy?
- What were the financial impacts of Ford's investments in Rivian and Argo in the
year 2022?
pipeline_tag: sentence-similarity
model-index:
- name: BGE-M3 Financial Matryoshka
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 1024
type: dim_1024
metrics:
- type: cosine_accuracy@1
value: 0.7171428571428572
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8314285714285714
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.87
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9142857142857143
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.7171428571428572
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.27714285714285714
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.174
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09142857142857141
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7171428571428572
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8314285714285714
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.87
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9142857142857143
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8152097277196483
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7835873015873015
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7867088346410263
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 768
type: dim_768
metrics:
- type: cosine_accuracy@1
value: 0.7128571428571429
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8342857142857143
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8657142857142858
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.91
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.7128571428571429
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2780952380952381
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.17314285714285713
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09099999999999998
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7128571428571429
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8342857142857143
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8657142857142858
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.91
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8122143155463835
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7808730158730155
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7843065190190194
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 512
type: dim_512
metrics:
- type: cosine_accuracy@1
value: 0.7114285714285714
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8357142857142857
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8642857142857143
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.91
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.7114285714285714
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2785714285714286
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.17285714285714285
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09099999999999998
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7114285714285714
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8357142857142857
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8642857142857143
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.91
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8109635546819154
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7792959183673466
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.782703758965192
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 384
type: dim_384
metrics:
- type: cosine_accuracy@1
value: 0.7142857142857143
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.8328571428571429
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.8628571428571429
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.9128571428571428
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.7142857142857143
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.2776190476190476
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.17257142857142854
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.09128571428571428
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.7142857142857143
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.8328571428571429
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.8628571428571429
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.9128571428571428
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.8125530857386527
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.7806292517006799
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.7837508100457361
name: Cosine Map@100
---
# BGE-M3 Financial Matryoshka
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
## Model Details
### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) <!-- at revision babcf60cae0a1f438d7ade582983d4ba462303c2 -->
- **Maximum Sequence Length:** 8192 tokens
- **Output Dimensionality:** 1024 tokens
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
- **Language:** en
- **License:** apache-2.0
### Model Sources
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
### Full Model Architecture
```
SentenceTransformer(
(0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
```
## Usage
### Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
```bash
pip install -U sentence-transformers
```
Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("haophancs/bge-m3-financial-matryoshka")
# Run inference
sentences = [
'As of January 28, 2024 the net carrying value of our inventories was $1.3 billion, which included provisions for obsolete and damaged inventory of $139.7 million.',
"What is the status of the company's inventory as of January 28, 2024, in terms of its valuation and provisions for obsolescence?",
'What is the relationship between the ESG goals and the long-term growth strategy?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```
<!--
### Direct Usage (Transformers)
<details><summary>Click to see the direct usage in Transformers</summary>
</details>
-->
<!--
### Downstream Usage (Sentence Transformers)
You can finetune this model on your own dataset.
<details><summary>Click to expand</summary>
</details>
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
## Evaluation
### Metrics
#### Information Retrieval
* Dataset: `dim_1024`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.7171 |
| cosine_accuracy@3 | 0.8314 |
| cosine_accuracy@5 | 0.87 |
| cosine_accuracy@10 | 0.9143 |
| cosine_precision@1 | 0.7171 |
| cosine_precision@3 | 0.2771 |
| cosine_precision@5 | 0.174 |
| cosine_precision@10 | 0.0914 |
| cosine_recall@1 | 0.7171 |
| cosine_recall@3 | 0.8314 |
| cosine_recall@5 | 0.87 |
| cosine_recall@10 | 0.9143 |
| cosine_ndcg@10 | 0.8152 |
| cosine_mrr@10 | 0.7836 |
| **cosine_map@100** | **0.7867** |
#### Information Retrieval
* Dataset: `dim_768`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.7129 |
| cosine_accuracy@3 | 0.8343 |
| cosine_accuracy@5 | 0.8657 |
| cosine_accuracy@10 | 0.91 |
| cosine_precision@1 | 0.7129 |
| cosine_precision@3 | 0.2781 |
| cosine_precision@5 | 0.1731 |
| cosine_precision@10 | 0.091 |
| cosine_recall@1 | 0.7129 |
| cosine_recall@3 | 0.8343 |
| cosine_recall@5 | 0.8657 |
| cosine_recall@10 | 0.91 |
| cosine_ndcg@10 | 0.8122 |
| cosine_mrr@10 | 0.7809 |
| **cosine_map@100** | **0.7843** |
#### Information Retrieval
* Dataset: `dim_512`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.7114 |
| cosine_accuracy@3 | 0.8357 |
| cosine_accuracy@5 | 0.8643 |
| cosine_accuracy@10 | 0.91 |
| cosine_precision@1 | 0.7114 |
| cosine_precision@3 | 0.2786 |
| cosine_precision@5 | 0.1729 |
| cosine_precision@10 | 0.091 |
| cosine_recall@1 | 0.7114 |
| cosine_recall@3 | 0.8357 |
| cosine_recall@5 | 0.8643 |
| cosine_recall@10 | 0.91 |
| cosine_ndcg@10 | 0.811 |
| cosine_mrr@10 | 0.7793 |
| **cosine_map@100** | **0.7827** |
#### Information Retrieval
* Dataset: `dim_384`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
| Metric | Value |
|:--------------------|:-----------|
| cosine_accuracy@1 | 0.7143 |
| cosine_accuracy@3 | 0.8329 |
| cosine_accuracy@5 | 0.8629 |
| cosine_accuracy@10 | 0.9129 |
| cosine_precision@1 | 0.7143 |
| cosine_precision@3 | 0.2776 |
| cosine_precision@5 | 0.1726 |
| cosine_precision@10 | 0.0913 |
| cosine_recall@1 | 0.7143 |
| cosine_recall@3 | 0.8329 |
| cosine_recall@5 | 0.8629 |
| cosine_recall@10 | 0.9129 |
| cosine_ndcg@10 | 0.8126 |
| cosine_mrr@10 | 0.7806 |
| **cosine_map@100** | **0.7838** |
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Dataset
#### Unnamed Dataset
* Size: 6,300 training samples
* Columns: <code>positive</code> and <code>anchor</code>
* Approximate statistics based on the first 1000 samples:
| | positive | anchor |
|:--------|:-------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------|
| type | string | string |
| details | <ul><li>min: 11 tokens</li><li>mean: 51.97 tokens</li><li>max: 1146 tokens</li></ul> | <ul><li>min: 7 tokens</li><li>mean: 21.63 tokens</li><li>max: 47 tokens</li></ul> |
* Samples:
| positive | anchor |
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------|
| <code>From fiscal year 2022 to 2023, the cost of revenue as a percentage of total net revenue decreased by 3 percent.</code> | <code>What was the percentage change in cost of revenue as a percentage of total net revenue from fiscal year 2022 to 2023?</code> |
| <code> •Operating income increased $321 million, or 2%, to $18.1 billion versus year ago due to the increase in net sales, partially offset by a modest decrease in operating margin.</code> | <code>What factors contributed to the increase in operating income for Procter & Gamble in 2023?</code> |
| <code>market specific brands including 'Aurrera,' 'Lider,' and 'PhonePe.'</code> | <code>What specific brands does Walmart International market?</code> |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
```json
{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
1024,
768,
512,
384
],
"matryoshka_weights": [
1,
1,
1,
1
],
"n_dims_per_step": -1
}
```
### Training Hyperparameters
#### Non-Default Hyperparameters
- `eval_strategy`: epoch
- `per_device_train_batch_size`: 4
- `per_device_eval_batch_size`: 2
- `gradient_accumulation_steps`: 2
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `batch_sampler`: no_duplicates
#### All Hyperparameters
<details><summary>Click to expand</summary>
- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 4
- `per_device_eval_batch_size`: 2
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 2
- `eval_accumulation_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: False
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`:
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional
</details>
### Training Logs
<details><summary>Click to expand</summary>
| Epoch | Step | Training Loss | dim_1024_cosine_map@100 | dim_384_cosine_map@100 | dim_512_cosine_map@100 | dim_768_cosine_map@100 |
|:----------:|:--------:|:-------------:|:-----------------------:|:----------------------:|:----------------------:|:----------------------:|
| 0.0127 | 10 | 0.2059 | - | - | - | - |
| 0.0254 | 20 | 0.2612 | - | - | - | - |
| 0.0381 | 30 | 0.0873 | - | - | - | - |
| 0.0508 | 40 | 0.1352 | - | - | - | - |
| 0.0635 | 50 | 0.156 | - | - | - | - |
| 0.0762 | 60 | 0.0407 | - | - | - | - |
| 0.0889 | 70 | 0.09 | - | - | - | - |
| 0.1016 | 80 | 0.027 | - | - | - | - |
| 0.1143 | 90 | 0.0978 | - | - | - | - |
| 0.1270 | 100 | 0.0105 | - | - | - | - |
| 0.1397 | 110 | 0.0402 | - | - | - | - |
| 0.1524 | 120 | 0.0745 | - | - | - | - |
| 0.1651 | 130 | 0.0655 | - | - | - | - |
| 0.1778 | 140 | 0.0075 | - | - | - | - |
| 0.1905 | 150 | 0.0141 | - | - | - | - |
| 0.2032 | 160 | 0.0615 | - | - | - | - |
| 0.2159 | 170 | 0.0029 | - | - | - | - |
| 0.2286 | 180 | 0.0269 | - | - | - | - |
| 0.2413 | 190 | 0.0724 | - | - | - | - |
| 0.2540 | 200 | 0.0218 | - | - | - | - |
| 0.2667 | 210 | 0.0027 | - | - | - | - |
| 0.2794 | 220 | 0.007 | - | - | - | - |
| 0.2921 | 230 | 0.0814 | - | - | - | - |
| 0.3048 | 240 | 0.0326 | - | - | - | - |
| 0.3175 | 250 | 0.0061 | - | - | - | - |
| 0.3302 | 260 | 0.0471 | - | - | - | - |
| 0.3429 | 270 | 0.0115 | - | - | - | - |
| 0.3556 | 280 | 0.0021 | - | - | - | - |
| 0.3683 | 290 | 0.0975 | - | - | - | - |
| 0.3810 | 300 | 0.0572 | - | - | - | - |
| 0.3937 | 310 | 0.0125 | - | - | - | - |
| 0.4063 | 320 | 0.04 | - | - | - | - |
| 0.4190 | 330 | 0.0023 | - | - | - | - |
| 0.4317 | 340 | 0.0121 | - | - | - | - |
| 0.4444 | 350 | 0.0116 | - | - | - | - |
| 0.4571 | 360 | 0.0059 | - | - | - | - |
| 0.4698 | 370 | 0.0217 | - | - | - | - |
| 0.4825 | 380 | 0.0294 | - | - | - | - |
| 0.4952 | 390 | 0.1102 | - | - | - | - |
| 0.5079 | 400 | 0.0103 | - | - | - | - |
| 0.5206 | 410 | 0.0023 | - | - | - | - |
| 0.5333 | 420 | 0.0157 | - | - | - | - |
| 0.5460 | 430 | 0.0805 | - | - | - | - |
| 0.5587 | 440 | 0.0168 | - | - | - | - |
| 0.5714 | 450 | 0.1279 | - | - | - | - |
| 0.5841 | 460 | 0.2012 | - | - | - | - |
| 0.5968 | 470 | 0.0436 | - | - | - | - |
| 0.6095 | 480 | 0.0204 | - | - | - | - |
| 0.6222 | 490 | 0.0097 | - | - | - | - |
| 0.6349 | 500 | 0.0013 | - | - | - | - |
| 0.6476 | 510 | 0.0042 | - | - | - | - |
| 0.6603 | 520 | 0.0034 | - | - | - | - |
| 0.6730 | 530 | 0.0226 | - | - | - | - |
| 0.6857 | 540 | 0.0267 | - | - | - | - |
| 0.6984 | 550 | 0.0007 | - | - | - | - |
| 0.7111 | 560 | 0.0766 | - | - | - | - |
| 0.7238 | 570 | 0.2174 | - | - | - | - |
| 0.7365 | 580 | 0.0089 | - | - | - | - |
| 0.7492 | 590 | 0.0794 | - | - | - | - |
| 0.7619 | 600 | 0.0031 | - | - | - | - |
| 0.7746 | 610 | 0.0499 | - | - | - | - |
| 0.7873 | 620 | 0.0105 | - | - | - | - |
| 0.8 | 630 | 0.0097 | - | - | - | - |
| 0.8127 | 640 | 0.0028 | - | - | - | - |
| 0.8254 | 650 | 0.0029 | - | - | - | - |
| 0.8381 | 660 | 0.1811 | - | - | - | - |
| 0.8508 | 670 | 0.064 | - | - | - | - |
| 0.8635 | 680 | 0.0139 | - | - | - | - |
| 0.8762 | 690 | 0.055 | - | - | - | - |
| 0.8889 | 700 | 0.0013 | - | - | - | - |
| 0.9016 | 710 | 0.0402 | - | - | - | - |
| 0.9143 | 720 | 0.0824 | - | - | - | - |
| 0.9270 | 730 | 0.03 | - | - | - | - |
| 0.9397 | 740 | 0.0337 | - | - | - | - |
| 0.9524 | 750 | 0.1192 | - | - | - | - |
| 0.9651 | 760 | 0.0039 | - | - | - | - |
| 0.9778 | 770 | 0.004 | - | - | - | - |
| 0.9905 | 780 | 0.1413 | - | - | - | - |
| 0.9994 | 787 | - | 0.7851 | 0.7794 | 0.7822 | 0.7863 |
| 1.0032 | 790 | 0.019 | - | - | - | - |
| 1.0159 | 800 | 0.0587 | - | - | - | - |
| 1.0286 | 810 | 0.0186 | - | - | - | - |
| 1.0413 | 820 | 0.0018 | - | - | - | - |
| 1.0540 | 830 | 0.0631 | - | - | - | - |
| 1.0667 | 840 | 0.0127 | - | - | - | - |
| 1.0794 | 850 | 0.0037 | - | - | - | - |
| 1.0921 | 860 | 0.0029 | - | - | - | - |
| 1.1048 | 870 | 0.1437 | - | - | - | - |
| 1.1175 | 880 | 0.0015 | - | - | - | - |
| 1.1302 | 890 | 0.0024 | - | - | - | - |
| 1.1429 | 900 | 0.0133 | - | - | - | - |
| 1.1556 | 910 | 0.0245 | - | - | - | - |
| 1.1683 | 920 | 0.0017 | - | - | - | - |
| 1.1810 | 930 | 0.0007 | - | - | - | - |
| 1.1937 | 940 | 0.002 | - | - | - | - |
| 1.2063 | 950 | 0.0044 | - | - | - | - |
| 1.2190 | 960 | 0.0009 | - | - | - | - |
| 1.2317 | 970 | 0.01 | - | - | - | - |
| 1.2444 | 980 | 0.0026 | - | - | - | - |
| 1.2571 | 990 | 0.0017 | - | - | - | - |
| 1.2698 | 1000 | 0.0014 | - | - | - | - |
| 1.2825 | 1010 | 0.0009 | - | - | - | - |
| 1.2952 | 1020 | 0.0829 | - | - | - | - |
| 1.3079 | 1030 | 0.0011 | - | - | - | - |
| 1.3206 | 1040 | 0.012 | - | - | - | - |
| 1.3333 | 1050 | 0.0019 | - | - | - | - |
| 1.3460 | 1060 | 0.0007 | - | - | - | - |
| 1.3587 | 1070 | 0.0141 | - | - | - | - |
| 1.3714 | 1080 | 0.0003 | - | - | - | - |
| 1.3841 | 1090 | 0.001 | - | - | - | - |
| 1.3968 | 1100 | 0.0005 | - | - | - | - |
| 1.4095 | 1110 | 0.0031 | - | - | - | - |
| 1.4222 | 1120 | 0.0004 | - | - | - | - |
| 1.4349 | 1130 | 0.0054 | - | - | - | - |
| 1.4476 | 1140 | 0.0003 | - | - | - | - |
| 1.4603 | 1150 | 0.0007 | - | - | - | - |
| 1.4730 | 1160 | 0.0009 | - | - | - | - |
| 1.4857 | 1170 | 0.001 | - | - | - | - |
| 1.4984 | 1180 | 0.0006 | - | - | - | - |
| 1.5111 | 1190 | 0.0046 | - | - | - | - |
| 1.5238 | 1200 | 0.0003 | - | - | - | - |
| 1.5365 | 1210 | 0.0002 | - | - | - | - |
| 1.5492 | 1220 | 0.004 | - | - | - | - |
| 1.5619 | 1230 | 0.0017 | - | - | - | - |
| 1.5746 | 1240 | 0.0003 | - | - | - | - |
| 1.5873 | 1250 | 0.0027 | - | - | - | - |
| 1.6 | 1260 | 0.1134 | - | - | - | - |
| 1.6127 | 1270 | 0.0007 | - | - | - | - |
| 1.6254 | 1280 | 0.0005 | - | - | - | - |
| 1.6381 | 1290 | 0.0008 | - | - | - | - |
| 1.6508 | 1300 | 0.0001 | - | - | - | - |
| 1.6635 | 1310 | 0.0023 | - | - | - | - |
| 1.6762 | 1320 | 0.0005 | - | - | - | - |
| 1.6889 | 1330 | 0.0004 | - | - | - | - |
| 1.7016 | 1340 | 0.0003 | - | - | - | - |
| 1.7143 | 1350 | 0.0347 | - | - | - | - |
| 1.7270 | 1360 | 0.0339 | - | - | - | - |
| 1.7397 | 1370 | 0.0003 | - | - | - | - |
| 1.7524 | 1380 | 0.0005 | - | - | - | - |
| 1.7651 | 1390 | 0.0002 | - | - | - | - |
| 1.7778 | 1400 | 0.0031 | - | - | - | - |
| 1.7905 | 1410 | 0.0002 | - | - | - | - |
| 1.8032 | 1420 | 0.0012 | - | - | - | - |
| 1.8159 | 1430 | 0.0002 | - | - | - | - |
| 1.8286 | 1440 | 0.0002 | - | - | - | - |
| 1.8413 | 1450 | 0.0004 | - | - | - | - |
| 1.8540 | 1460 | 0.011 | - | - | - | - |
| 1.8667 | 1470 | 0.0824 | - | - | - | - |
| 1.8794 | 1480 | 0.0003 | - | - | - | - |
| 1.8921 | 1490 | 0.0004 | - | - | - | - |
| 1.9048 | 1500 | 0.0006 | - | - | - | - |
| 1.9175 | 1510 | 0.015 | - | - | - | - |
| 1.9302 | 1520 | 0.0004 | - | - | - | - |
| 1.9429 | 1530 | 0.0004 | - | - | - | - |
| 1.9556 | 1540 | 0.0011 | - | - | - | - |
| 1.9683 | 1550 | 0.0003 | - | - | - | - |
| 1.9810 | 1560 | 0.0006 | - | - | - | - |
| 1.9937 | 1570 | 0.0042 | - | - | - | - |
| 2.0 | 1575 | - | 0.7862 | 0.7855 | 0.7852 | 0.7878 |
| 2.0063 | 1580 | 0.0005 | - | - | - | - |
| 2.0190 | 1590 | 0.002 | - | - | - | - |
| 2.0317 | 1600 | 0.0013 | - | - | - | - |
| 2.0444 | 1610 | 0.0002 | - | - | - | - |
| 2.0571 | 1620 | 0.0035 | - | - | - | - |
| 2.0698 | 1630 | 0.0004 | - | - | - | - |
| 2.0825 | 1640 | 0.0002 | - | - | - | - |
| 2.0952 | 1650 | 0.0032 | - | - | - | - |
| 2.1079 | 1660 | 0.0916 | - | - | - | - |
| 2.1206 | 1670 | 0.0002 | - | - | - | - |
| 2.1333 | 1680 | 0.0006 | - | - | - | - |
| 2.1460 | 1690 | 0.0002 | - | - | - | - |
| 2.1587 | 1700 | 0.0003 | - | - | - | - |
| 2.1714 | 1710 | 0.0001 | - | - | - | - |
| 2.1841 | 1720 | 0.0001 | - | - | - | - |
| 2.1968 | 1730 | 0.0004 | - | - | - | - |
| 2.2095 | 1740 | 0.0004 | - | - | - | - |
| 2.2222 | 1750 | 0.0001 | - | - | - | - |
| 2.2349 | 1760 | 0.0002 | - | - | - | - |
| 2.2476 | 1770 | 0.0007 | - | - | - | - |
| 2.2603 | 1780 | 0.0001 | - | - | - | - |
| 2.2730 | 1790 | 0.0002 | - | - | - | - |
| 2.2857 | 1800 | 0.0004 | - | - | - | - |
| 2.2984 | 1810 | 0.0711 | - | - | - | - |
| 2.3111 | 1820 | 0.0001 | - | - | - | - |
| 2.3238 | 1830 | 0.0005 | - | - | - | - |
| 2.3365 | 1840 | 0.0004 | - | - | - | - |
| 2.3492 | 1850 | 0.0001 | - | - | - | - |
| 2.3619 | 1860 | 0.0005 | - | - | - | - |
| 2.3746 | 1870 | 0.0003 | - | - | - | - |
| 2.3873 | 1880 | 0.0001 | - | - | - | - |
| 2.4 | 1890 | 0.0002 | - | - | - | - |
| 2.4127 | 1900 | 0.0001 | - | - | - | - |
| 2.4254 | 1910 | 0.0002 | - | - | - | - |
| 2.4381 | 1920 | 0.0002 | - | - | - | - |
| 2.4508 | 1930 | 0.0002 | - | - | - | - |
| 2.4635 | 1940 | 0.0004 | - | - | - | - |
| 2.4762 | 1950 | 0.0001 | - | - | - | - |
| 2.4889 | 1960 | 0.0002 | - | - | - | - |
| 2.5016 | 1970 | 0.0002 | - | - | - | - |
| 2.5143 | 1980 | 0.0001 | - | - | - | - |
| 2.5270 | 1990 | 0.0001 | - | - | - | - |
| 2.5397 | 2000 | 0.0002 | - | - | - | - |
| 2.5524 | 2010 | 0.0023 | - | - | - | - |
| 2.5651 | 2020 | 0.0002 | - | - | - | - |
| 2.5778 | 2030 | 0.0001 | - | - | - | - |
| 2.5905 | 2040 | 0.0003 | - | - | - | - |
| 2.6032 | 2050 | 0.0003 | - | - | - | - |
| 2.6159 | 2060 | 0.0002 | - | - | - | - |
| 2.6286 | 2070 | 0.0001 | - | - | - | - |
| 2.6413 | 2080 | 0.0 | - | - | - | - |
| 2.6540 | 2090 | 0.0001 | - | - | - | - |
| 2.6667 | 2100 | 0.0001 | - | - | - | - |
| 2.6794 | 2110 | 0.0001 | - | - | - | - |
| 2.6921 | 2120 | 0.0001 | - | - | - | - |
| 2.7048 | 2130 | 0.0001 | - | - | - | - |
| 2.7175 | 2140 | 0.0048 | - | - | - | - |
| 2.7302 | 2150 | 0.0005 | - | - | - | - |
| 2.7429 | 2160 | 0.0001 | - | - | - | - |
| 2.7556 | 2170 | 0.0001 | - | - | - | - |
| 2.7683 | 2180 | 0.0001 | - | - | - | - |
| 2.7810 | 2190 | 0.0001 | - | - | - | - |
| 2.7937 | 2200 | 0.0001 | - | - | - | - |
| 2.8063 | 2210 | 0.0001 | - | - | - | - |
| 2.8190 | 2220 | 0.0001 | - | - | - | - |
| 2.8317 | 2230 | 0.0002 | - | - | - | - |
| 2.8444 | 2240 | 0.0036 | - | - | - | - |
| 2.8571 | 2250 | 0.0001 | - | - | - | - |
| 2.8698 | 2260 | 0.0368 | - | - | - | - |
| 2.8825 | 2270 | 0.0003 | - | - | - | - |
| 2.8952 | 2280 | 0.0002 | - | - | - | - |
| 2.9079 | 2290 | 0.0001 | - | - | - | - |
| 2.9206 | 2300 | 0.0005 | - | - | - | - |
| 2.9333 | 2310 | 0.0001 | - | - | - | - |
| 2.9460 | 2320 | 0.0001 | - | - | - | - |
| 2.9587 | 2330 | 0.0003 | - | - | - | - |
| 2.9714 | 2340 | 0.0001 | - | - | - | - |
| 2.9841 | 2350 | 0.0001 | - | - | - | - |
| 2.9968 | 2360 | 0.0002 | - | - | - | - |
| **2.9994** | **2362** | **-** | **0.7864** | **0.7805** | **0.7838** | **0.7852** |
| 3.0095 | 2370 | 0.0025 | - | - | - | - |
| 3.0222 | 2380 | 0.0002 | - | - | - | - |
| 3.0349 | 2390 | 0.0001 | - | - | - | - |
| 3.0476 | 2400 | 0.0001 | - | - | - | - |
| 3.0603 | 2410 | 0.0001 | - | - | - | - |
| 3.0730 | 2420 | 0.0001 | - | - | - | - |
| 3.0857 | 2430 | 0.0001 | - | - | - | - |
| 3.0984 | 2440 | 0.0002 | - | - | - | - |
| 3.1111 | 2450 | 0.0116 | - | - | - | - |
| 3.1238 | 2460 | 0.0002 | - | - | - | - |
| 3.1365 | 2470 | 0.0001 | - | - | - | - |
| 3.1492 | 2480 | 0.0001 | - | - | - | - |
| 3.1619 | 2490 | 0.0001 | - | - | - | - |
| 3.1746 | 2500 | 0.0001 | - | - | - | - |
| 3.1873 | 2510 | 0.0001 | - | - | - | - |
| 3.2 | 2520 | 0.0001 | - | - | - | - |
| 3.2127 | 2530 | 0.0001 | - | - | - | - |
| 3.2254 | 2540 | 0.0001 | - | - | - | - |
| 3.2381 | 2550 | 0.0002 | - | - | - | - |
| 3.2508 | 2560 | 0.0001 | - | - | - | - |
| 3.2635 | 2570 | 0.0001 | - | - | - | - |
| 3.2762 | 2580 | 0.0001 | - | - | - | - |
| 3.2889 | 2590 | 0.0001 | - | - | - | - |
| 3.3016 | 2600 | 0.063 | - | - | - | - |
| 3.3143 | 2610 | 0.0001 | - | - | - | - |
| 3.3270 | 2620 | 0.0001 | - | - | - | - |
| 3.3397 | 2630 | 0.0001 | - | - | - | - |
| 3.3524 | 2640 | 0.0001 | - | - | - | - |
| 3.3651 | 2650 | 0.0002 | - | - | - | - |
| 3.3778 | 2660 | 0.0001 | - | - | - | - |
| 3.3905 | 2670 | 0.0001 | - | - | - | - |
| 3.4032 | 2680 | 0.0001 | - | - | - | - |
| 3.4159 | 2690 | 0.0001 | - | - | - | - |
| 3.4286 | 2700 | 0.0001 | - | - | - | - |
| 3.4413 | 2710 | 0.0001 | - | - | - | - |
| 3.4540 | 2720 | 0.0002 | - | - | - | - |
| 3.4667 | 2730 | 0.0001 | - | - | - | - |
| 3.4794 | 2740 | 0.0001 | - | - | - | - |
| 3.4921 | 2750 | 0.0001 | - | - | - | - |
| 3.5048 | 2760 | 0.0001 | - | - | - | - |
| 3.5175 | 2770 | 0.0002 | - | - | - | - |
| 3.5302 | 2780 | 0.0001 | - | - | - | - |
| 3.5429 | 2790 | 0.0001 | - | - | - | - |
| 3.5556 | 2800 | 0.0001 | - | - | - | - |
| 3.5683 | 2810 | 0.0001 | - | - | - | - |
| 3.5810 | 2820 | 0.0001 | - | - | - | - |
| 3.5937 | 2830 | 0.0001 | - | - | - | - |
| 3.6063 | 2840 | 0.0001 | - | - | - | - |
| 3.6190 | 2850 | 0.0 | - | - | - | - |
| 3.6317 | 2860 | 0.0001 | - | - | - | - |
| 3.6444 | 2870 | 0.0001 | - | - | - | - |
| 3.6571 | 2880 | 0.0001 | - | - | - | - |
| 3.6698 | 2890 | 0.0001 | - | - | - | - |
| 3.6825 | 2900 | 0.0001 | - | - | - | - |
| 3.6952 | 2910 | 0.0001 | - | - | - | - |
| 3.7079 | 2920 | 0.0001 | - | - | - | - |
| 3.7206 | 2930 | 0.0003 | - | - | - | - |
| 3.7333 | 2940 | 0.0001 | - | - | - | - |
| 3.7460 | 2950 | 0.0001 | - | - | - | - |
| 3.7587 | 2960 | 0.0001 | - | - | - | - |
| 3.7714 | 2970 | 0.0002 | - | - | - | - |
| 3.7841 | 2980 | 0.0001 | - | - | - | - |
| 3.7968 | 2990 | 0.0001 | - | - | - | - |
| 3.8095 | 3000 | 0.0001 | - | - | - | - |
| 3.8222 | 3010 | 0.0001 | - | - | - | - |
| 3.8349 | 3020 | 0.0002 | - | - | - | - |
| 3.8476 | 3030 | 0.0001 | - | - | - | - |
| 3.8603 | 3040 | 0.0001 | - | - | - | - |
| 3.8730 | 3050 | 0.0214 | - | - | - | - |
| 3.8857 | 3060 | 0.0001 | - | - | - | - |
| 3.8984 | 3070 | 0.0001 | - | - | - | - |
| 3.9111 | 3080 | 0.0001 | - | - | - | - |
| 3.9238 | 3090 | 0.0001 | - | - | - | - |
| 3.9365 | 3100 | 0.0001 | - | - | - | - |
| 3.9492 | 3110 | 0.0001 | - | - | - | - |
| 3.9619 | 3120 | 0.0001 | - | - | - | - |
| 3.9746 | 3130 | 0.0001 | - | - | - | - |
| 3.9873 | 3140 | 0.0001 | - | - | - | - |
| 3.9975 | 3148 | - | 0.7867 | 0.7838 | 0.7827 | 0.7843 |
* The bold row denotes the saved checkpoint.
</details>
### Framework Versions
- Python: 3.12.2
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.2.0+cu121
- Accelerate: 0.31.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1
## Citation
### BibTeX
#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
```
#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
```
#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->