metadata

language:
  - en
license: apache-2.0
library_name: sentence-transformers
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:6300
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-m3
datasets: []
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
widget:
  - source_sentence: >-
      The consolidated financial statements and accompanying notes listed in
      Part IV, Item 15(a)(1) of this Annual Report on Form 10-K.
    sentences:
      - >-
        How much total space does an average The Home Depot store encompass
        including its garden area?
      - >-
        What section of the Annual Report on Form 10-K contains the consolidated
        financial statements and accompanying notes?
      - >-
        What types of competitive factors does Garmin believe are important in
        its markets?
  - source_sentence: >-
      Item 3. Legal Proceedings, which covers litigation and regulatory matters,
      refers to Note 12 – Commitments and Contingencies for more detailed
      information within the Consolidated Financial Statements.
    sentences:
      - >-
        What pages contain the Financial Statements and Supplementary Data in
        IBM’s 2023 Annual Report to Stockholders?
      - >-
        In which note can further details on Legal Proceedings be found within
        the Consolidated Financial Statements?
      - What is the title of Item 8 in the document?
  - source_sentence: Net Revenues for the Entertainment segment were $659.3 million in 2023.
    sentences:
      - What were the net revenues for the Entertainment segment in 2023?
      - How much net cash was provided by operating activities in 2023?
      - >-
        What was the net income reported for the fiscal year ending in August
        2023?
  - source_sentence: >-
      The capital allocation program focuses on three objectives: (1) grow our
      business at an average target ROIC-adjusted rate of 20% or greater; (2)
      maintain a strong investment-grade balance sheet, including a target
      average automotive cash balance of $18.0 billion; and (3) after the first
      two objectives are met, return available cash to shareholders.
    sentences:
      - >-
        Why is ICE Mortgage Technology subject to the examination by the Federal
        Financial Institutions Examination Council (FFIEC) and its member
        agencies?
      - >-
        What type of regulations do U.S. automobiles need to comply with under
        the National Highway Traffic Safety Administration?
      - >-
        What are the three objectives of the capital allocation program
        referenced?
  - source_sentence: >-
      As of January 28, 2024 the net carrying value of our inventories was $1.3
      billion, which included provisions for obsolete and damaged inventory of
      $139.7 million.
    sentences:
      - >-
        What is the status of the company's inventory as of January 28, 2024, in
        terms of its valuation and provisions for obsolescence?
      - >-
        What is the relationship between the ESG goals and the long-term growth
        strategy?
      - >-
        What were the financial impacts of Ford's investments in Rivian and Argo
        in the year 2022?
pipeline_tag: sentence-similarity
model-index:
  - name: BGE-M3 Financial Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 1024
          type: dim_1024
        metrics:
          - type: cosine_accuracy@1
            value: 0.7171428571428572
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8314285714285714
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.87
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9142857142857143
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7171428571428572
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.27714285714285714
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.174
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09142857142857141
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7171428571428572
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8314285714285714
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.87
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9142857142857143
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8152097277196483
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7835873015873015
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7867088346410263
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.7128571428571429
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8342857142857143
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8657142857142858
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.91
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7128571428571429
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2780952380952381
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17314285714285713
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09099999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7128571428571429
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8342857142857143
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8657142857142858
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.91
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8122143155463835
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7808730158730155
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7843065190190194
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.7114285714285714
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8357142857142857
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8642857142857143
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.91
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7114285714285714
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2785714285714286
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17285714285714285
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09099999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7114285714285714
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8357142857142857
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8642857142857143
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.91
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8109635546819154
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7792959183673466
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.782703758965192
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 384
          type: dim_384
        metrics:
          - type: cosine_accuracy@1
            value: 0.7142857142857143
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8328571428571429
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.8628571428571429
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9128571428571428
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7142857142857143
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2776190476190476
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.17257142857142854
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09128571428571428
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7142857142857143
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8328571428571429
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.8628571428571429
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9128571428571428
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8125530857386527
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.7806292517006799
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.7837508100457361
            name: Cosine Map@100

BGE-M3 Financial Matryoshka

This is a sentence-transformers model finetuned from BAAI/bge-m3. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-m3
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 1024 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("haophancs/bge-m3-financial-matryoshka")
# Run inference
sentences = [
    'As of January 28, 2024 the net carrying value of our inventories was $1.3 billion, which included provisions for obsolete and damaged inventory of $139.7 million.',
    "What is the status of the company's inventory as of January 28, 2024, in terms of its valuation and provisions for obsolescence?",
    'What is the relationship between the ESG goals and the long-term growth strategy?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_1024
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7171
cosine_accuracy@3	0.8314
cosine_accuracy@5	0.87
cosine_accuracy@10	0.9143
cosine_precision@1	0.7171
cosine_precision@3	0.2771
cosine_precision@5	0.174
cosine_precision@10	0.0914
cosine_recall@1	0.7171
cosine_recall@3	0.8314
cosine_recall@5	0.87
cosine_recall@10	0.9143
cosine_ndcg@10	0.8152
cosine_mrr@10	0.7836
cosine_map@100	0.7867

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7129
cosine_accuracy@3	0.8343
cosine_accuracy@5	0.8657
cosine_accuracy@10	0.91
cosine_precision@1	0.7129
cosine_precision@3	0.2781
cosine_precision@5	0.1731
cosine_precision@10	0.091
cosine_recall@1	0.7129
cosine_recall@3	0.8343
cosine_recall@5	0.8657
cosine_recall@10	0.91
cosine_ndcg@10	0.8122
cosine_mrr@10	0.7809
cosine_map@100	0.7843

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7114
cosine_accuracy@3	0.8357
cosine_accuracy@5	0.8643
cosine_accuracy@10	0.91
cosine_precision@1	0.7114
cosine_precision@3	0.2786
cosine_precision@5	0.1729
cosine_precision@10	0.091
cosine_recall@1	0.7114
cosine_recall@3	0.8357
cosine_recall@5	0.8643
cosine_recall@10	0.91
cosine_ndcg@10	0.811
cosine_mrr@10	0.7793
cosine_map@100	0.7827

Information Retrieval

Dataset: dim_384
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7143
cosine_accuracy@3	0.8329
cosine_accuracy@5	0.8629
cosine_accuracy@10	0.9129
cosine_precision@1	0.7143
cosine_precision@3	0.2776
cosine_precision@5	0.1726
cosine_precision@10	0.0913
cosine_recall@1	0.7143
cosine_recall@3	0.8329
cosine_recall@5	0.8629
cosine_recall@10	0.9129
cosine_ndcg@10	0.8126
cosine_mrr@10	0.7806
cosine_map@100	0.7838

Training Details

Training Dataset

Unnamed Dataset

Size: 6,300 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 11 tokens
mean: 51.97 tokens
max: 1146 tokens

min: 7 tokens
mean: 21.63 tokens
max: 47 tokens

	positive	anchor
type	string	string
details	min: 11 tokens mean: 51.97 tokens max: 1146 tokens	min: 7 tokens mean: 21.63 tokens max: 47 tokens

Samples:

positive	anchor
`From fiscal year 2022 to 2023, the cost of revenue as a percentage of total net revenue decreased by 3 percent.`	`What was the percentage change in cost of revenue as a percentage of total net revenue from fiscal year 2022 to 2023?`
`•Operating income increased $321 million, or 2%, to $18.1 billion versus year ago due to the increase in net sales, partially offset by a modest decrease in operating margin.`	`What factors contributed to the increase in operating income for Procter & Gamble in 2023?`
`market specific brands including 'Aurrera,' 'Lider,' and 'PhonePe.'`	`What specific brands does Walmart International market?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        1024,
        768,
        512,
        384
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 4
per_device_eval_batch_size: 2
gradient_accumulation_steps: 2
learning_rate: 2e-05
num_train_epochs: 4
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 2
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 2
eval_accumulation_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: True
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	dim_1024_cosine_map@100	dim_384_cosine_map@100	dim_512_cosine_map@100	dim_768_cosine_map@100
0.0127	10	0.2059	-	-	-	-
0.0254	20	0.2612	-	-	-	-
0.0381	30	0.0873	-	-	-	-
0.0508	40	0.1352	-	-	-	-
0.0635	50	0.156	-	-	-	-
0.0762	60	0.0407	-	-	-	-
0.0889	70	0.09	-	-	-	-
0.1016	80	0.027	-	-	-	-
0.1143	90	0.0978	-	-	-	-
0.1270	100	0.0105	-	-	-	-
0.1397	110	0.0402	-	-	-	-
0.1524	120	0.0745	-	-	-	-
0.1651	130	0.0655	-	-	-	-
0.1778	140	0.0075	-	-	-	-
0.1905	150	0.0141	-	-	-	-
0.2032	160	0.0615	-	-	-	-
0.2159	170	0.0029	-	-	-	-
0.2286	180	0.0269	-	-	-	-
0.2413	190	0.0724	-	-	-	-
0.2540	200	0.0218	-	-	-	-
0.2667	210	0.0027	-	-	-	-
0.2794	220	0.007	-	-	-	-
0.2921	230	0.0814	-	-	-	-
0.3048	240	0.0326	-	-	-	-
0.3175	250	0.0061	-	-	-	-
0.3302	260	0.0471	-	-	-	-
0.3429	270	0.0115	-	-	-	-
0.3556	280	0.0021	-	-	-	-
0.3683	290	0.0975	-	-	-	-
0.3810	300	0.0572	-	-	-	-
0.3937	310	0.0125	-	-	-	-
0.4063	320	0.04	-	-	-	-
0.4190	330	0.0023	-	-	-	-
0.4317	340	0.0121	-	-	-	-
0.4444	350	0.0116	-	-	-	-
0.4571	360	0.0059	-	-	-	-
0.4698	370	0.0217	-	-	-	-
0.4825	380	0.0294	-	-	-	-
0.4952	390	0.1102	-	-	-	-
0.5079	400	0.0103	-	-	-	-
0.5206	410	0.0023	-	-	-	-
0.5333	420	0.0157	-	-	-	-
0.5460	430	0.0805	-	-	-	-
0.5587	440	0.0168	-	-	-	-
0.5714	450	0.1279	-	-	-	-
0.5841	460	0.2012	-	-	-	-
0.5968	470	0.0436	-	-	-	-
0.6095	480	0.0204	-	-	-	-
0.6222	490	0.0097	-	-	-	-
0.6349	500	0.0013	-	-	-	-
0.6476	510	0.0042	-	-	-	-
0.6603	520	0.0034	-	-	-	-
0.6730	530	0.0226	-	-	-	-
0.6857	540	0.0267	-	-	-	-
0.6984	550	0.0007	-	-	-	-
0.7111	560	0.0766	-	-	-	-
0.7238	570	0.2174	-	-	-	-
0.7365	580	0.0089	-	-	-	-
0.7492	590	0.0794	-	-	-	-
0.7619	600	0.0031	-	-	-	-
0.7746	610	0.0499	-	-	-	-
0.7873	620	0.0105	-	-	-	-
0.8	630	0.0097	-	-	-	-
0.8127	640	0.0028	-	-	-	-
0.8254	650	0.0029	-	-	-	-
0.8381	660	0.1811	-	-	-	-
0.8508	670	0.064	-	-	-	-
0.8635	680	0.0139	-	-	-	-
0.8762	690	0.055	-	-	-	-
0.8889	700	0.0013	-	-	-	-
0.9016	710	0.0402	-	-	-	-
0.9143	720	0.0824	-	-	-	-
0.9270	730	0.03	-	-	-	-
0.9397	740	0.0337	-	-	-	-
0.9524	750	0.1192	-	-	-	-
0.9651	760	0.0039	-	-	-	-
0.9778	770	0.004	-	-	-	-
0.9905	780	0.1413	-	-	-	-
0.9994	787	-	0.7851	0.7794	0.7822	0.7863
1.0032	790	0.019	-	-	-	-
1.0159	800	0.0587	-	-	-	-
1.0286	810	0.0186	-	-	-	-
1.0413	820	0.0018	-	-	-	-
1.0540	830	0.0631	-	-	-	-
1.0667	840	0.0127	-	-	-	-
1.0794	850	0.0037	-	-	-	-
1.0921	860	0.0029	-	-	-	-
1.1048	870	0.1437	-	-	-	-
1.1175	880	0.0015	-	-	-	-
1.1302	890	0.0024	-	-	-	-
1.1429	900	0.0133	-	-	-	-
1.1556	910	0.0245	-	-	-	-
1.1683	920	0.0017	-	-	-	-
1.1810	930	0.0007	-	-	-	-
1.1937	940	0.002	-	-	-	-
1.2063	950	0.0044	-	-	-	-
1.2190	960	0.0009	-	-	-	-
1.2317	970	0.01	-	-	-	-
1.2444	980	0.0026	-	-	-	-
1.2571	990	0.0017	-	-	-	-
1.2698	1000	0.0014	-	-	-	-
1.2825	1010	0.0009	-	-	-	-
1.2952	1020	0.0829	-	-	-	-
1.3079	1030	0.0011	-	-	-	-
1.3206	1040	0.012	-	-	-	-
1.3333	1050	0.0019	-	-	-	-
1.3460	1060	0.0007	-	-	-	-
1.3587	1070	0.0141	-	-	-	-
1.3714	1080	0.0003	-	-	-	-
1.3841	1090	0.001	-	-	-	-
1.3968	1100	0.0005	-	-	-	-
1.4095	1110	0.0031	-	-	-	-
1.4222	1120	0.0004	-	-	-	-
1.4349	1130	0.0054	-	-	-	-
1.4476	1140	0.0003	-	-	-	-
1.4603	1150	0.0007	-	-	-	-
1.4730	1160	0.0009	-	-	-	-
1.4857	1170	0.001	-	-	-	-
1.4984	1180	0.0006	-	-	-	-
1.5111	1190	0.0046	-	-	-	-
1.5238	1200	0.0003	-	-	-	-
1.5365	1210	0.0002	-	-	-	-
1.5492	1220	0.004	-	-	-	-
1.5619	1230	0.0017	-	-	-	-
1.5746	1240	0.0003	-	-	-	-
1.5873	1250	0.0027	-	-	-	-
1.6	1260	0.1134	-	-	-	-
1.6127	1270	0.0007	-	-	-	-
1.6254	1280	0.0005	-	-	-	-
1.6381	1290	0.0008	-	-	-	-
1.6508	1300	0.0001	-	-	-	-
1.6635	1310	0.0023	-	-	-	-
1.6762	1320	0.0005	-	-	-	-
1.6889	1330	0.0004	-	-	-	-
1.7016	1340	0.0003	-	-	-	-
1.7143	1350	0.0347	-	-	-	-
1.7270	1360	0.0339	-	-	-	-
1.7397	1370	0.0003	-	-	-	-
1.7524	1380	0.0005	-	-	-	-
1.7651	1390	0.0002	-	-	-	-
1.7778	1400	0.0031	-	-	-	-
1.7905	1410	0.0002	-	-	-	-
1.8032	1420	0.0012	-	-	-	-
1.8159	1430	0.0002	-	-	-	-
1.8286	1440	0.0002	-	-	-	-
1.8413	1450	0.0004	-	-	-	-
1.8540	1460	0.011	-	-	-	-
1.8667	1470	0.0824	-	-	-	-
1.8794	1480	0.0003	-	-	-	-
1.8921	1490	0.0004	-	-	-	-
1.9048	1500	0.0006	-	-	-	-
1.9175	1510	0.015	-	-	-	-
1.9302	1520	0.0004	-	-	-	-
1.9429	1530	0.0004	-	-	-	-
1.9556	1540	0.0011	-	-	-	-
1.9683	1550	0.0003	-	-	-	-
1.9810	1560	0.0006	-	-	-	-
1.9937	1570	0.0042	-	-	-	-
2.0	1575	-	0.7862	0.7855	0.7852	0.7878
2.0063	1580	0.0005	-	-	-	-
2.0190	1590	0.002	-	-	-	-
2.0317	1600	0.0013	-	-	-	-
2.0444	1610	0.0002	-	-	-	-
2.0571	1620	0.0035	-	-	-	-
2.0698	1630	0.0004	-	-	-	-
2.0825	1640	0.0002	-	-	-	-
2.0952	1650	0.0032	-	-	-	-
2.1079	1660	0.0916	-	-	-	-
2.1206	1670	0.0002	-	-	-	-
2.1333	1680	0.0006	-	-	-	-
2.1460	1690	0.0002	-	-	-	-
2.1587	1700	0.0003	-	-	-	-
2.1714	1710	0.0001	-	-	-	-
2.1841	1720	0.0001	-	-	-	-
2.1968	1730	0.0004	-	-	-	-
2.2095	1740	0.0004	-	-	-	-
2.2222	1750	0.0001	-	-	-	-
2.2349	1760	0.0002	-	-	-	-
2.2476	1770	0.0007	-	-	-	-
2.2603	1780	0.0001	-	-	-	-
2.2730	1790	0.0002	-	-	-	-
2.2857	1800	0.0004	-	-	-	-
2.2984	1810	0.0711	-	-	-	-
2.3111	1820	0.0001	-	-	-	-
2.3238	1830	0.0005	-	-	-	-
2.3365	1840	0.0004	-	-	-	-
2.3492	1850	0.0001	-	-	-	-
2.3619	1860	0.0005	-	-	-	-
2.3746	1870	0.0003	-	-	-	-
2.3873	1880	0.0001	-	-	-	-
2.4	1890	0.0002	-	-	-	-
2.4127	1900	0.0001	-	-	-	-
2.4254	1910	0.0002	-	-	-	-
2.4381	1920	0.0002	-	-	-	-
2.4508	1930	0.0002	-	-	-	-
2.4635	1940	0.0004	-	-	-	-
2.4762	1950	0.0001	-	-	-	-
2.4889	1960	0.0002	-	-	-	-
2.5016	1970	0.0002	-	-	-	-
2.5143	1980	0.0001	-	-	-	-
2.5270	1990	0.0001	-	-	-	-
2.5397	2000	0.0002	-	-	-	-
2.5524	2010	0.0023	-	-	-	-
2.5651	2020	0.0002	-	-	-	-
2.5778	2030	0.0001	-	-	-	-
2.5905	2040	0.0003	-	-	-	-
2.6032	2050	0.0003	-	-	-	-
2.6159	2060	0.0002	-	-	-	-
2.6286	2070	0.0001	-	-	-	-
2.6413	2080	0.0	-	-	-	-
2.6540	2090	0.0001	-	-	-	-
2.6667	2100	0.0001	-	-	-	-
2.6794	2110	0.0001	-	-	-	-
2.6921	2120	0.0001	-	-	-	-
2.7048	2130	0.0001	-	-	-	-
2.7175	2140	0.0048	-	-	-	-
2.7302	2150	0.0005	-	-	-	-
2.7429	2160	0.0001	-	-	-	-
2.7556	2170	0.0001	-	-	-	-
2.7683	2180	0.0001	-	-	-	-
2.7810	2190	0.0001	-	-	-	-
2.7937	2200	0.0001	-	-	-	-
2.8063	2210	0.0001	-	-	-	-
2.8190	2220	0.0001	-	-	-	-
2.8317	2230	0.0002	-	-	-	-
2.8444	2240	0.0036	-	-	-	-
2.8571	2250	0.0001	-	-	-	-
2.8698	2260	0.0368	-	-	-	-
2.8825	2270	0.0003	-	-	-	-
2.8952	2280	0.0002	-	-	-	-
2.9079	2290	0.0001	-	-	-	-
2.9206	2300	0.0005	-	-	-	-
2.9333	2310	0.0001	-	-	-	-
2.9460	2320	0.0001	-	-	-	-
2.9587	2330	0.0003	-	-	-	-
2.9714	2340	0.0001	-	-	-	-
2.9841	2350	0.0001	-	-	-	-
2.9968	2360	0.0002	-	-	-	-
2.9994	2362	-	0.7864	0.7805	0.7838	0.7852
3.0095	2370	0.0025	-	-	-	-
3.0222	2380	0.0002	-	-	-	-
3.0349	2390	0.0001	-	-	-	-
3.0476	2400	0.0001	-	-	-	-
3.0603	2410	0.0001	-	-	-	-
3.0730	2420	0.0001	-	-	-	-
3.0857	2430	0.0001	-	-	-	-
3.0984	2440	0.0002	-	-	-	-
3.1111	2450	0.0116	-	-	-	-
3.1238	2460	0.0002	-	-	-	-
3.1365	2470	0.0001	-	-	-	-
3.1492	2480	0.0001	-	-	-	-
3.1619	2490	0.0001	-	-	-	-
3.1746	2500	0.0001	-	-	-	-
3.1873	2510	0.0001	-	-	-	-
3.2	2520	0.0001	-	-	-	-
3.2127	2530	0.0001	-	-	-	-
3.2254	2540	0.0001	-	-	-	-
3.2381	2550	0.0002	-	-	-	-
3.2508	2560	0.0001	-	-	-	-
3.2635	2570	0.0001	-	-	-	-
3.2762	2580	0.0001	-	-	-	-
3.2889	2590	0.0001	-	-	-	-
3.3016	2600	0.063	-	-	-	-
3.3143	2610	0.0001	-	-	-	-
3.3270	2620	0.0001	-	-	-	-
3.3397	2630	0.0001	-	-	-	-
3.3524	2640	0.0001	-	-	-	-
3.3651	2650	0.0002	-	-	-	-
3.3778	2660	0.0001	-	-	-	-
3.3905	2670	0.0001	-	-	-	-
3.4032	2680	0.0001	-	-	-	-
3.4159	2690	0.0001	-	-	-	-
3.4286	2700	0.0001	-	-	-	-
3.4413	2710	0.0001	-	-	-	-
3.4540	2720	0.0002	-	-	-	-
3.4667	2730	0.0001	-	-	-	-
3.4794	2740	0.0001	-	-	-	-
3.4921	2750	0.0001	-	-	-	-
3.5048	2760	0.0001	-	-	-	-
3.5175	2770	0.0002	-	-	-	-
3.5302	2780	0.0001	-	-	-	-
3.5429	2790	0.0001	-	-	-	-
3.5556	2800	0.0001	-	-	-	-
3.5683	2810	0.0001	-	-	-	-
3.5810	2820	0.0001	-	-	-	-
3.5937	2830	0.0001	-	-	-	-
3.6063	2840	0.0001	-	-	-	-
3.6190	2850	0.0	-	-	-	-
3.6317	2860	0.0001	-	-	-	-
3.6444	2870	0.0001	-	-	-	-
3.6571	2880	0.0001	-	-	-	-
3.6698	2890	0.0001	-	-	-	-
3.6825	2900	0.0001	-	-	-	-
3.6952	2910	0.0001	-	-	-	-
3.7079	2920	0.0001	-	-	-	-
3.7206	2930	0.0003	-	-	-	-
3.7333	2940	0.0001	-	-	-	-
3.7460	2950	0.0001	-	-	-	-
3.7587	2960	0.0001	-	-	-	-
3.7714	2970	0.0002	-	-	-	-
3.7841	2980	0.0001	-	-	-	-
3.7968	2990	0.0001	-	-	-	-
3.8095	3000	0.0001	-	-	-	-
3.8222	3010	0.0001	-	-	-	-
3.8349	3020	0.0002	-	-	-	-
3.8476	3030	0.0001	-	-	-	-
3.8603	3040	0.0001	-	-	-	-
3.8730	3050	0.0214	-	-	-	-
3.8857	3060	0.0001	-	-	-	-
3.8984	3070	0.0001	-	-	-	-
3.9111	3080	0.0001	-	-	-	-
3.9238	3090	0.0001	-	-	-	-
3.9365	3100	0.0001	-	-	-	-
3.9492	3110	0.0001	-	-	-	-
3.9619	3120	0.0001	-	-	-	-
3.9746	3130	0.0001	-	-	-	-
3.9873	3140	0.0001	-	-	-	-
3.9975	3148	-	0.7867	0.7838	0.7827	0.7843

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.2
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.2.0+cu121
Accelerate: 0.31.0
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}