--- language: - en license: apache-2.0 library_name: sentence-transformers tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:6300 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss base_model: BAAI/bge-m3 datasets: [] metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 widget: - source_sentence: The consolidated financial statements and accompanying notes listed in Part IV, Item 15(a)(1) of this Annual Report on Form 10-K. sentences: - How much total space does an average The Home Depot store encompass including its garden area? - What section of the Annual Report on Form 10-K contains the consolidated financial statements and accompanying notes? - What types of competitive factors does Garmin believe are important in its markets? - source_sentence: Item 3. Legal Proceedings, which covers litigation and regulatory matters, refers to Note 12 – Commitments and Contingencies for more detailed information within the Consolidated Financial Statements. sentences: - What pages contain the Financial Statements and Supplementary Data in IBM’s 2023 Annual Report to Stockholders? - In which note can further details on Legal Proceedings be found within the Consolidated Financial Statements? - What is the title of Item 8 in the document? - source_sentence: Net Revenues for the Entertainment segment were $659.3 million in 2023. sentences: - What were the net revenues for the Entertainment segment in 2023? - How much net cash was provided by operating activities in 2023? - What was the net income reported for the fiscal year ending in August 2023? - source_sentence: 'The capital allocation program focuses on three objectives: (1) grow our business at an average target ROIC-adjusted rate of 20% or greater; (2) maintain a strong investment-grade balance sheet, including a target average automotive cash balance of $18.0 billion; and (3) after the first two objectives are met, return available cash to shareholders.' sentences: - Why is ICE Mortgage Technology subject to the examination by the Federal Financial Institutions Examination Council (FFIEC) and its member agencies? - What type of regulations do U.S. automobiles need to comply with under the National Highway Traffic Safety Administration? - What are the three objectives of the capital allocation program referenced? - source_sentence: As of January 28, 2024 the net carrying value of our inventories was $1.3 billion, which included provisions for obsolete and damaged inventory of $139.7 million. sentences: - What is the status of the company's inventory as of January 28, 2024, in terms of its valuation and provisions for obsolescence? - What is the relationship between the ESG goals and the long-term growth strategy? - What were the financial impacts of Ford's investments in Rivian and Argo in the year 2022? pipeline_tag: sentence-similarity model-index: - name: BGE-M3 Financial Matryoshka results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 1024 type: dim_1024 metrics: - type: cosine_accuracy@1 value: 0.7171428571428572 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8314285714285714 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.87 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9142857142857143 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7171428571428572 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.27714285714285714 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.174 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09142857142857141 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7171428571428572 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8314285714285714 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.87 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9142857142857143 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8152097277196483 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7835873015873015 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7867088346410263 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 768 type: dim_768 metrics: - type: cosine_accuracy@1 value: 0.7128571428571429 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8342857142857143 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8657142857142858 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.91 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7128571428571429 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2780952380952381 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.17314285714285713 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09099999999999998 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7128571428571429 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8342857142857143 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8657142857142858 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.91 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8122143155463835 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7808730158730155 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7843065190190194 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 512 type: dim_512 metrics: - type: cosine_accuracy@1 value: 0.7114285714285714 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8357142857142857 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8642857142857143 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.91 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7114285714285714 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2785714285714286 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.17285714285714285 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09099999999999998 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7114285714285714 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8357142857142857 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8642857142857143 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.91 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8109635546819154 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7792959183673466 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.782703758965192 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 384 type: dim_384 metrics: - type: cosine_accuracy@1 value: 0.7142857142857143 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8328571428571429 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8628571428571429 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9128571428571428 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7142857142857143 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2776190476190476 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.17257142857142854 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09128571428571428 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7142857142857143 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8328571428571429 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8628571428571429 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9128571428571428 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8125530857386527 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7806292517006799 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7837508100457361 name: Cosine Map@100 --- # BGE-M3 Financial Matryoshka This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) - **Maximum Sequence Length:** 8192 tokens - **Output Dimensionality:** 1024 tokens - **Similarity Function:** Cosine Similarity - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("haophancs/bge-m3-financial-matryoshka") # Run inference sentences = [ 'As of January 28, 2024 the net carrying value of our inventories was $1.3 billion, which included provisions for obsolete and damaged inventory of $139.7 million.', "What is the status of the company's inventory as of January 28, 2024, in terms of its valuation and provisions for obsolescence?", 'What is the relationship between the ESG goals and the long-term growth strategy?', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 1024] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_1024` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7171 | | cosine_accuracy@3 | 0.8314 | | cosine_accuracy@5 | 0.87 | | cosine_accuracy@10 | 0.9143 | | cosine_precision@1 | 0.7171 | | cosine_precision@3 | 0.2771 | | cosine_precision@5 | 0.174 | | cosine_precision@10 | 0.0914 | | cosine_recall@1 | 0.7171 | | cosine_recall@3 | 0.8314 | | cosine_recall@5 | 0.87 | | cosine_recall@10 | 0.9143 | | cosine_ndcg@10 | 0.8152 | | cosine_mrr@10 | 0.7836 | | **cosine_map@100** | **0.7867** | #### Information Retrieval * Dataset: `dim_768` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7129 | | cosine_accuracy@3 | 0.8343 | | cosine_accuracy@5 | 0.8657 | | cosine_accuracy@10 | 0.91 | | cosine_precision@1 | 0.7129 | | cosine_precision@3 | 0.2781 | | cosine_precision@5 | 0.1731 | | cosine_precision@10 | 0.091 | | cosine_recall@1 | 0.7129 | | cosine_recall@3 | 0.8343 | | cosine_recall@5 | 0.8657 | | cosine_recall@10 | 0.91 | | cosine_ndcg@10 | 0.8122 | | cosine_mrr@10 | 0.7809 | | **cosine_map@100** | **0.7843** | #### Information Retrieval * Dataset: `dim_512` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7114 | | cosine_accuracy@3 | 0.8357 | | cosine_accuracy@5 | 0.8643 | | cosine_accuracy@10 | 0.91 | | cosine_precision@1 | 0.7114 | | cosine_precision@3 | 0.2786 | | cosine_precision@5 | 0.1729 | | cosine_precision@10 | 0.091 | | cosine_recall@1 | 0.7114 | | cosine_recall@3 | 0.8357 | | cosine_recall@5 | 0.8643 | | cosine_recall@10 | 0.91 | | cosine_ndcg@10 | 0.811 | | cosine_mrr@10 | 0.7793 | | **cosine_map@100** | **0.7827** | #### Information Retrieval * Dataset: `dim_384` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7143 | | cosine_accuracy@3 | 0.8329 | | cosine_accuracy@5 | 0.8629 | | cosine_accuracy@10 | 0.9129 | | cosine_precision@1 | 0.7143 | | cosine_precision@3 | 0.2776 | | cosine_precision@5 | 0.1726 | | cosine_precision@10 | 0.0913 | | cosine_recall@1 | 0.7143 | | cosine_recall@3 | 0.8329 | | cosine_recall@5 | 0.8629 | | cosine_recall@10 | 0.9129 | | cosine_ndcg@10 | 0.8126 | | cosine_mrr@10 | 0.7806 | | **cosine_map@100** | **0.7838** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 6,300 training samples * Columns: positive and anchor * Approximate statistics based on the first 1000 samples: | | positive | anchor | |:--------|:-------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | positive | anchor | |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------| | From fiscal year 2022 to 2023, the cost of revenue as a percentage of total net revenue decreased by 3 percent. | What was the percentage change in cost of revenue as a percentage of total net revenue from fiscal year 2022 to 2023? | | •Operating income increased $321 million, or 2%, to $18.1 billion versus year ago due to the increase in net sales, partially offset by a modest decrease in operating margin. | What factors contributed to the increase in operating income for Procter & Gamble in 2023? | | market specific brands including 'Aurrera,' 'Lider,' and 'PhonePe.' | What specific brands does Walmart International market? | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 1024, 768, 512, 384 ], "matryoshka_weights": [ 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 4 - `per_device_eval_batch_size`: 2 - `gradient_accumulation_steps`: 2 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `tf32`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 4 - `per_device_eval_batch_size`: 2 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 2 - `eval_accumulation_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs
Click to expand | Epoch | Step | Training Loss | dim_1024_cosine_map@100 | dim_384_cosine_map@100 | dim_512_cosine_map@100 | dim_768_cosine_map@100 | |:----------:|:--------:|:-------------:|:-----------------------:|:----------------------:|:----------------------:|:----------------------:| | 0.0127 | 10 | 0.2059 | - | - | - | - | | 0.0254 | 20 | 0.2612 | - | - | - | - | | 0.0381 | 30 | 0.0873 | - | - | - | - | | 0.0508 | 40 | 0.1352 | - | - | - | - | | 0.0635 | 50 | 0.156 | - | - | - | - | | 0.0762 | 60 | 0.0407 | - | - | - | - | | 0.0889 | 70 | 0.09 | - | - | - | - | | 0.1016 | 80 | 0.027 | - | - | - | - | | 0.1143 | 90 | 0.0978 | - | - | - | - | | 0.1270 | 100 | 0.0105 | - | - | - | - | | 0.1397 | 110 | 0.0402 | - | - | - | - | | 0.1524 | 120 | 0.0745 | - | - | - | - | | 0.1651 | 130 | 0.0655 | - | - | - | - | | 0.1778 | 140 | 0.0075 | - | - | - | - | | 0.1905 | 150 | 0.0141 | - | - | - | - | | 0.2032 | 160 | 0.0615 | - | - | - | - | | 0.2159 | 170 | 0.0029 | - | - | - | - | | 0.2286 | 180 | 0.0269 | - | - | - | - | | 0.2413 | 190 | 0.0724 | - | - | - | - | | 0.2540 | 200 | 0.0218 | - | - | - | - | | 0.2667 | 210 | 0.0027 | - | - | - | - | | 0.2794 | 220 | 0.007 | - | - | - | - | | 0.2921 | 230 | 0.0814 | - | - | - | - | | 0.3048 | 240 | 0.0326 | - | - | - | - | | 0.3175 | 250 | 0.0061 | - | - | - | - | | 0.3302 | 260 | 0.0471 | - | - | - | - | | 0.3429 | 270 | 0.0115 | - | - | - | - | | 0.3556 | 280 | 0.0021 | - | - | - | - | | 0.3683 | 290 | 0.0975 | - | - | - | - | | 0.3810 | 300 | 0.0572 | - | - | - | - | | 0.3937 | 310 | 0.0125 | - | - | - | - | | 0.4063 | 320 | 0.04 | - | - | - | - | | 0.4190 | 330 | 0.0023 | - | - | - | - | | 0.4317 | 340 | 0.0121 | - | - | - | - | | 0.4444 | 350 | 0.0116 | - | - | - | - | | 0.4571 | 360 | 0.0059 | - | - | - | - | | 0.4698 | 370 | 0.0217 | - | - | - | - | | 0.4825 | 380 | 0.0294 | - | - | - | - | | 0.4952 | 390 | 0.1102 | - | - | - | - | | 0.5079 | 400 | 0.0103 | - | - | - | - | | 0.5206 | 410 | 0.0023 | - | - | - | - | | 0.5333 | 420 | 0.0157 | - | - | - | - | | 0.5460 | 430 | 0.0805 | - | - | - | - | | 0.5587 | 440 | 0.0168 | - | - | - | - | | 0.5714 | 450 | 0.1279 | - | - | - | - | | 0.5841 | 460 | 0.2012 | - | - | - | - | | 0.5968 | 470 | 0.0436 | - | - | - | - | | 0.6095 | 480 | 0.0204 | - | - | - | - | | 0.6222 | 490 | 0.0097 | - | - | - | - | | 0.6349 | 500 | 0.0013 | - | - | - | - | | 0.6476 | 510 | 0.0042 | - | - | - | - | | 0.6603 | 520 | 0.0034 | - | - | - | - | | 0.6730 | 530 | 0.0226 | - | - | - | - | | 0.6857 | 540 | 0.0267 | - | - | - | - | | 0.6984 | 550 | 0.0007 | - | - | - | - | | 0.7111 | 560 | 0.0766 | - | - | - | - | | 0.7238 | 570 | 0.2174 | - | - | - | - | | 0.7365 | 580 | 0.0089 | - | - | - | - | | 0.7492 | 590 | 0.0794 | - | - | - | - | | 0.7619 | 600 | 0.0031 | - | - | - | - | | 0.7746 | 610 | 0.0499 | - | - | - | - | | 0.7873 | 620 | 0.0105 | - | - | - | - | | 0.8 | 630 | 0.0097 | - | - | - | - | | 0.8127 | 640 | 0.0028 | - | - | - | - | | 0.8254 | 650 | 0.0029 | - | - | - | - | | 0.8381 | 660 | 0.1811 | - | - | - | - | | 0.8508 | 670 | 0.064 | - | - | - | - | | 0.8635 | 680 | 0.0139 | - | - | - | - | | 0.8762 | 690 | 0.055 | - | - | - | - | | 0.8889 | 700 | 0.0013 | - | - | - | - | | 0.9016 | 710 | 0.0402 | - | - | - | - | | 0.9143 | 720 | 0.0824 | - | - | - | - | | 0.9270 | 730 | 0.03 | - | - | - | - | | 0.9397 | 740 | 0.0337 | - | - | - | - | | 0.9524 | 750 | 0.1192 | - | - | - | - | | 0.9651 | 760 | 0.0039 | - | - | - | - | | 0.9778 | 770 | 0.004 | - | - | - | - | | 0.9905 | 780 | 0.1413 | - | - | - | - | | 0.9994 | 787 | - | 0.7851 | 0.7794 | 0.7822 | 0.7863 | | 1.0032 | 790 | 0.019 | - | - | - | - | | 1.0159 | 800 | 0.0587 | - | - | - | - | | 1.0286 | 810 | 0.0186 | - | - | - | - | | 1.0413 | 820 | 0.0018 | - | - | - | - | | 1.0540 | 830 | 0.0631 | - | - | - | - | | 1.0667 | 840 | 0.0127 | - | - | - | - | | 1.0794 | 850 | 0.0037 | - | - | - | - | | 1.0921 | 860 | 0.0029 | - | - | - | - | | 1.1048 | 870 | 0.1437 | - | - | - | - | | 1.1175 | 880 | 0.0015 | - | - | - | - | | 1.1302 | 890 | 0.0024 | - | - | - | - | | 1.1429 | 900 | 0.0133 | - | - | - | - | | 1.1556 | 910 | 0.0245 | - | - | - | - | | 1.1683 | 920 | 0.0017 | - | - | - | - | | 1.1810 | 930 | 0.0007 | - | - | - | - | | 1.1937 | 940 | 0.002 | - | - | - | - | | 1.2063 | 950 | 0.0044 | - | - | - | - | | 1.2190 | 960 | 0.0009 | - | - | - | - | | 1.2317 | 970 | 0.01 | - | - | - | - | | 1.2444 | 980 | 0.0026 | - | - | - | - | | 1.2571 | 990 | 0.0017 | - | - | - | - | | 1.2698 | 1000 | 0.0014 | - | - | - | - | | 1.2825 | 1010 | 0.0009 | - | - | - | - | | 1.2952 | 1020 | 0.0829 | - | - | - | - | | 1.3079 | 1030 | 0.0011 | - | - | - | - | | 1.3206 | 1040 | 0.012 | - | - | - | - | | 1.3333 | 1050 | 0.0019 | - | - | - | - | | 1.3460 | 1060 | 0.0007 | - | - | - | - | | 1.3587 | 1070 | 0.0141 | - | - | - | - | | 1.3714 | 1080 | 0.0003 | - | - | - | - | | 1.3841 | 1090 | 0.001 | - | - | - | - | | 1.3968 | 1100 | 0.0005 | - | - | - | - | | 1.4095 | 1110 | 0.0031 | - | - | - | - | | 1.4222 | 1120 | 0.0004 | - | - | - | - | | 1.4349 | 1130 | 0.0054 | - | - | - | - | | 1.4476 | 1140 | 0.0003 | - | - | - | - | | 1.4603 | 1150 | 0.0007 | - | - | - | - | | 1.4730 | 1160 | 0.0009 | - | - | - | - | | 1.4857 | 1170 | 0.001 | - | - | - | - | | 1.4984 | 1180 | 0.0006 | - | - | - | - | | 1.5111 | 1190 | 0.0046 | - | - | - | - | | 1.5238 | 1200 | 0.0003 | - | - | - | - | | 1.5365 | 1210 | 0.0002 | - | - | - | - | | 1.5492 | 1220 | 0.004 | - | - | - | - | | 1.5619 | 1230 | 0.0017 | - | - | - | - | | 1.5746 | 1240 | 0.0003 | - | - | - | - | | 1.5873 | 1250 | 0.0027 | - | - | - | - | | 1.6 | 1260 | 0.1134 | - | - | - | - | | 1.6127 | 1270 | 0.0007 | - | - | - | - | | 1.6254 | 1280 | 0.0005 | - | - | - | - | | 1.6381 | 1290 | 0.0008 | - | - | - | - | | 1.6508 | 1300 | 0.0001 | - | - | - | - | | 1.6635 | 1310 | 0.0023 | - | - | - | - | | 1.6762 | 1320 | 0.0005 | - | - | - | - | | 1.6889 | 1330 | 0.0004 | - | - | - | - | | 1.7016 | 1340 | 0.0003 | - | - | - | - | | 1.7143 | 1350 | 0.0347 | - | - | - | - | | 1.7270 | 1360 | 0.0339 | - | - | - | - | | 1.7397 | 1370 | 0.0003 | - | - | - | - | | 1.7524 | 1380 | 0.0005 | - | - | - | - | | 1.7651 | 1390 | 0.0002 | - | - | - | - | | 1.7778 | 1400 | 0.0031 | - | - | - | - | | 1.7905 | 1410 | 0.0002 | - | - | - | - | | 1.8032 | 1420 | 0.0012 | - | - | - | - | | 1.8159 | 1430 | 0.0002 | - | - | - | - | | 1.8286 | 1440 | 0.0002 | - | - | - | - | | 1.8413 | 1450 | 0.0004 | - | - | - | - | | 1.8540 | 1460 | 0.011 | - | - | - | - | | 1.8667 | 1470 | 0.0824 | - | - | - | - | | 1.8794 | 1480 | 0.0003 | - | - | - | - | | 1.8921 | 1490 | 0.0004 | - | - | - | - | | 1.9048 | 1500 | 0.0006 | - | - | - | - | | 1.9175 | 1510 | 0.015 | - | - | - | - | | 1.9302 | 1520 | 0.0004 | - | - | - | - | | 1.9429 | 1530 | 0.0004 | - | - | - | - | | 1.9556 | 1540 | 0.0011 | - | - | - | - | | 1.9683 | 1550 | 0.0003 | - | - | - | - | | 1.9810 | 1560 | 0.0006 | - | - | - | - | | 1.9937 | 1570 | 0.0042 | - | - | - | - | | 2.0 | 1575 | - | 0.7862 | 0.7855 | 0.7852 | 0.7878 | | 2.0063 | 1580 | 0.0005 | - | - | - | - | | 2.0190 | 1590 | 0.002 | - | - | - | - | | 2.0317 | 1600 | 0.0013 | - | - | - | - | | 2.0444 | 1610 | 0.0002 | - | - | - | - | | 2.0571 | 1620 | 0.0035 | - | - | - | - | | 2.0698 | 1630 | 0.0004 | - | - | - | - | | 2.0825 | 1640 | 0.0002 | - | - | - | - | | 2.0952 | 1650 | 0.0032 | - | - | - | - | | 2.1079 | 1660 | 0.0916 | - | - | - | - | | 2.1206 | 1670 | 0.0002 | - | - | - | - | | 2.1333 | 1680 | 0.0006 | - | - | - | - | | 2.1460 | 1690 | 0.0002 | - | - | - | - | | 2.1587 | 1700 | 0.0003 | - | - | - | - | | 2.1714 | 1710 | 0.0001 | - | - | - | - | | 2.1841 | 1720 | 0.0001 | - | - | - | - | | 2.1968 | 1730 | 0.0004 | - | - | - | - | | 2.2095 | 1740 | 0.0004 | - | - | - | - | | 2.2222 | 1750 | 0.0001 | - | - | - | - | | 2.2349 | 1760 | 0.0002 | - | - | - | - | | 2.2476 | 1770 | 0.0007 | - | - | - | - | | 2.2603 | 1780 | 0.0001 | - | - | - | - | | 2.2730 | 1790 | 0.0002 | - | - | - | - | | 2.2857 | 1800 | 0.0004 | - | - | - | - | | 2.2984 | 1810 | 0.0711 | - | - | - | - | | 2.3111 | 1820 | 0.0001 | - | - | - | - | | 2.3238 | 1830 | 0.0005 | - | - | - | - | | 2.3365 | 1840 | 0.0004 | - | - | - | - | | 2.3492 | 1850 | 0.0001 | - | - | - | - | | 2.3619 | 1860 | 0.0005 | - | - | - | - | | 2.3746 | 1870 | 0.0003 | - | - | - | - | | 2.3873 | 1880 | 0.0001 | - | - | - | - | | 2.4 | 1890 | 0.0002 | - | - | - | - | | 2.4127 | 1900 | 0.0001 | - | - | - | - | | 2.4254 | 1910 | 0.0002 | - | - | - | - | | 2.4381 | 1920 | 0.0002 | - | - | - | - | | 2.4508 | 1930 | 0.0002 | - | - | - | - | | 2.4635 | 1940 | 0.0004 | - | - | - | - | | 2.4762 | 1950 | 0.0001 | - | - | - | - | | 2.4889 | 1960 | 0.0002 | - | - | - | - | | 2.5016 | 1970 | 0.0002 | - | - | - | - | | 2.5143 | 1980 | 0.0001 | - | - | - | - | | 2.5270 | 1990 | 0.0001 | - | - | - | - | | 2.5397 | 2000 | 0.0002 | - | - | - | - | | 2.5524 | 2010 | 0.0023 | - | - | - | - | | 2.5651 | 2020 | 0.0002 | - | - | - | - | | 2.5778 | 2030 | 0.0001 | - | - | - | - | | 2.5905 | 2040 | 0.0003 | - | - | - | - | | 2.6032 | 2050 | 0.0003 | - | - | - | - | | 2.6159 | 2060 | 0.0002 | - | - | - | - | | 2.6286 | 2070 | 0.0001 | - | - | - | - | | 2.6413 | 2080 | 0.0 | - | - | - | - | | 2.6540 | 2090 | 0.0001 | - | - | - | - | | 2.6667 | 2100 | 0.0001 | - | - | - | - | | 2.6794 | 2110 | 0.0001 | - | - | - | - | | 2.6921 | 2120 | 0.0001 | - | - | - | - | | 2.7048 | 2130 | 0.0001 | - | - | - | - | | 2.7175 | 2140 | 0.0048 | - | - | - | - | | 2.7302 | 2150 | 0.0005 | - | - | - | - | | 2.7429 | 2160 | 0.0001 | - | - | - | - | | 2.7556 | 2170 | 0.0001 | - | - | - | - | | 2.7683 | 2180 | 0.0001 | - | - | - | - | | 2.7810 | 2190 | 0.0001 | - | - | - | - | | 2.7937 | 2200 | 0.0001 | - | - | - | - | | 2.8063 | 2210 | 0.0001 | - | - | - | - | | 2.8190 | 2220 | 0.0001 | - | - | - | - | | 2.8317 | 2230 | 0.0002 | - | - | - | - | | 2.8444 | 2240 | 0.0036 | - | - | - | - | | 2.8571 | 2250 | 0.0001 | - | - | - | - | | 2.8698 | 2260 | 0.0368 | - | - | - | - | | 2.8825 | 2270 | 0.0003 | - | - | - | - | | 2.8952 | 2280 | 0.0002 | - | - | - | - | | 2.9079 | 2290 | 0.0001 | - | - | - | - | | 2.9206 | 2300 | 0.0005 | - | - | - | - | | 2.9333 | 2310 | 0.0001 | - | - | - | - | | 2.9460 | 2320 | 0.0001 | - | - | - | - | | 2.9587 | 2330 | 0.0003 | - | - | - | - | | 2.9714 | 2340 | 0.0001 | - | - | - | - | | 2.9841 | 2350 | 0.0001 | - | - | - | - | | 2.9968 | 2360 | 0.0002 | - | - | - | - | | **2.9994** | **2362** | **-** | **0.7864** | **0.7805** | **0.7838** | **0.7852** | | 3.0095 | 2370 | 0.0025 | - | - | - | - | | 3.0222 | 2380 | 0.0002 | - | - | - | - | | 3.0349 | 2390 | 0.0001 | - | - | - | - | | 3.0476 | 2400 | 0.0001 | - | - | - | - | | 3.0603 | 2410 | 0.0001 | - | - | - | - | | 3.0730 | 2420 | 0.0001 | - | - | - | - | | 3.0857 | 2430 | 0.0001 | - | - | - | - | | 3.0984 | 2440 | 0.0002 | - | - | - | - | | 3.1111 | 2450 | 0.0116 | - | - | - | - | | 3.1238 | 2460 | 0.0002 | - | - | - | - | | 3.1365 | 2470 | 0.0001 | - | - | - | - | | 3.1492 | 2480 | 0.0001 | - | - | - | - | | 3.1619 | 2490 | 0.0001 | - | - | - | - | | 3.1746 | 2500 | 0.0001 | - | - | - | - | | 3.1873 | 2510 | 0.0001 | - | - | - | - | | 3.2 | 2520 | 0.0001 | - | - | - | - | | 3.2127 | 2530 | 0.0001 | - | - | - | - | | 3.2254 | 2540 | 0.0001 | - | - | - | - | | 3.2381 | 2550 | 0.0002 | - | - | - | - | | 3.2508 | 2560 | 0.0001 | - | - | - | - | | 3.2635 | 2570 | 0.0001 | - | - | - | - | | 3.2762 | 2580 | 0.0001 | - | - | - | - | | 3.2889 | 2590 | 0.0001 | - | - | - | - | | 3.3016 | 2600 | 0.063 | - | - | - | - | | 3.3143 | 2610 | 0.0001 | - | - | - | - | | 3.3270 | 2620 | 0.0001 | - | - | - | - | | 3.3397 | 2630 | 0.0001 | - | - | - | - | | 3.3524 | 2640 | 0.0001 | - | - | - | - | | 3.3651 | 2650 | 0.0002 | - | - | - | - | | 3.3778 | 2660 | 0.0001 | - | - | - | - | | 3.3905 | 2670 | 0.0001 | - | - | - | - | | 3.4032 | 2680 | 0.0001 | - | - | - | - | | 3.4159 | 2690 | 0.0001 | - | - | - | - | | 3.4286 | 2700 | 0.0001 | - | - | - | - | | 3.4413 | 2710 | 0.0001 | - | - | - | - | | 3.4540 | 2720 | 0.0002 | - | - | - | - | | 3.4667 | 2730 | 0.0001 | - | - | - | - | | 3.4794 | 2740 | 0.0001 | - | - | - | - | | 3.4921 | 2750 | 0.0001 | - | - | - | - | | 3.5048 | 2760 | 0.0001 | - | - | - | - | | 3.5175 | 2770 | 0.0002 | - | - | - | - | | 3.5302 | 2780 | 0.0001 | - | - | - | - | | 3.5429 | 2790 | 0.0001 | - | - | - | - | | 3.5556 | 2800 | 0.0001 | - | - | - | - | | 3.5683 | 2810 | 0.0001 | - | - | - | - | | 3.5810 | 2820 | 0.0001 | - | - | - | - | | 3.5937 | 2830 | 0.0001 | - | - | - | - | | 3.6063 | 2840 | 0.0001 | - | - | - | - | | 3.6190 | 2850 | 0.0 | - | - | - | - | | 3.6317 | 2860 | 0.0001 | - | - | - | - | | 3.6444 | 2870 | 0.0001 | - | - | - | - | | 3.6571 | 2880 | 0.0001 | - | - | - | - | | 3.6698 | 2890 | 0.0001 | - | - | - | - | | 3.6825 | 2900 | 0.0001 | - | - | - | - | | 3.6952 | 2910 | 0.0001 | - | - | - | - | | 3.7079 | 2920 | 0.0001 | - | - | - | - | | 3.7206 | 2930 | 0.0003 | - | - | - | - | | 3.7333 | 2940 | 0.0001 | - | - | - | - | | 3.7460 | 2950 | 0.0001 | - | - | - | - | | 3.7587 | 2960 | 0.0001 | - | - | - | - | | 3.7714 | 2970 | 0.0002 | - | - | - | - | | 3.7841 | 2980 | 0.0001 | - | - | - | - | | 3.7968 | 2990 | 0.0001 | - | - | - | - | | 3.8095 | 3000 | 0.0001 | - | - | - | - | | 3.8222 | 3010 | 0.0001 | - | - | - | - | | 3.8349 | 3020 | 0.0002 | - | - | - | - | | 3.8476 | 3030 | 0.0001 | - | - | - | - | | 3.8603 | 3040 | 0.0001 | - | - | - | - | | 3.8730 | 3050 | 0.0214 | - | - | - | - | | 3.8857 | 3060 | 0.0001 | - | - | - | - | | 3.8984 | 3070 | 0.0001 | - | - | - | - | | 3.9111 | 3080 | 0.0001 | - | - | - | - | | 3.9238 | 3090 | 0.0001 | - | - | - | - | | 3.9365 | 3100 | 0.0001 | - | - | - | - | | 3.9492 | 3110 | 0.0001 | - | - | - | - | | 3.9619 | 3120 | 0.0001 | - | - | - | - | | 3.9746 | 3130 | 0.0001 | - | - | - | - | | 3.9873 | 3140 | 0.0001 | - | - | - | - | | 3.9975 | 3148 | - | 0.7867 | 0.7838 | 0.7827 | 0.7843 | * The bold row denotes the saved checkpoint.
### Framework Versions - Python: 3.12.2 - Sentence Transformers: 3.0.1 - Transformers: 4.41.2 - PyTorch: 2.2.0+cu121 - Accelerate: 0.31.0 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```