--- base_model: BAAI/bge-base-en-v1.5 datasets: [] language: - en library_name: sentence-transformers license: apache-2.0 metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 pipeline_tag: sentence-similarity tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:6300 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss widget: - source_sentence: The sales contracts for Israel contain formulas that generally reflect an initial base price subject to price indexation, Brent-linked or other, over the life of the contract. sentences: - What was the change in HP's net deferred tax assets from 2022 to 2023? - What are the pricing mechanisms for crude oil sales contracts in Israel? - What was the total net income tax benefit HP received related to foreign tax audit matters? - source_sentence: The FCA imposes severe penalties for the knowing and improper retention of overpayments from government programs. In addition, the defendant must follow certain notification and repayment processes within 60 days of identifying and quantifying an overpayment. sentences: - What does Note 21 pertain to in this report? - What types of penalties does the FCA impose for the knowing and improper retention of overpayments from government payors? - What impact did discrete tax items have on the tax provision in 2023 compared to 2022? - source_sentence: The expected long-term rate of return is evaluated on an annual basis. We consider a number of factors when setting assumptions with respect to the long-term rate of return, including current and expected asset allocation and historical and expected returns on the plan asset categories. Actual asset allocations are regularly reviewed and periodically rebalanced to the targeted allocations when considered appropriate. sentences: - How is the expected long-term rate of return on plan assets determined? - What is the accumulated benefit obligation for AT&T's pension plans as of December 31, 2023? - What is the management philosophy of Johnson & Johnson known as? - source_sentence: The functional currency of our foreign entities is the currency of the primary economic environment in which the entity operates. sentences: - By what percent did Other Income (Expense) change in 2023 compared to 2022? - What are the Canadian class actions against Equifax seeking in relation to the 2017 cybersecurity incident? - What is the functional currency for a company's foreign entities? - source_sentence: Our products compete with other commercially available products based primarily on efficacy, safety, tolerability, acceptance by doctors, ease of patient compliance, ease of use, price, insurance and other reimbursement coverage, distribution and marketing. sentences: - What are the main factors influencing competition for the company's products? - What was the impact of restructuring charges in 2022 on the company and what changes occurred in 2023? - What are the penalties for non-compliance with Brazil's data protection laws? model-index: - name: BGE base Financial Matryoshka results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 768 type: dim_768 metrics: - type: cosine_accuracy@1 value: 0.6985714285714286 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.83 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.88 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9257142857142857 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.6985714285714286 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.27666666666666667 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.176 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09257142857142854 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.6985714285714286 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.83 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.88 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9257142857142857 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8141629079228132 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7782318594104309 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7807867705374557 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 512 type: dim_512 metrics: - type: cosine_accuracy@1 value: 0.7014285714285714 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8328571428571429 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8857142857142857 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9228571428571428 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7014285714285714 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2776190476190476 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.17714285714285713 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09228571428571428 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7014285714285714 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8328571428571429 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8857142857142857 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9228571428571428 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8133531244983723 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7781366213151925 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7808747462599953 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 0.7 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.84 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8714285714285714 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9085714285714286 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.28 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.17428571428571427 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09085714285714284 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.84 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8714285714285714 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9085714285714286 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8077154994184018 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7749937641723353 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7785241448057054 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 0.6942857142857143 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.82 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8557142857142858 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9028571428571428 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.6942857142857143 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.2733333333333333 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.17114285714285712 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09028571428571427 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.6942857142857143 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.82 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8557142857142858 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9028571428571428 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.7990640908671799 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7658554421768706 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7697199109144424 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.6614285714285715 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.7842857142857143 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8271428571428572 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8885714285714286 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.6614285714285715 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.26142857142857145 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.1654285714285714 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08885714285714284 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.6614285714285715 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.7842857142857143 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8271428571428572 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.8885714285714286 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.7730930913085324 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7365589569160996 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7404183138657333 name: Cosine Map@100 --- # BGE base Financial Matryoshka This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 tokens - **Similarity Function:** Cosine Similarity - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("felipehsilveira/bge-base-financial-matryoshka") # Run inference sentences = [ 'Our products compete with other commercially available products based primarily on efficacy, safety, tolerability, acceptance by doctors, ease of patient compliance, ease of use, price, insurance and other reimbursement coverage, distribution and marketing.', "What are the main factors influencing competition for the company's products?", 'What was the impact of restructuring charges in 2022 on the company and what changes occurred in 2023?', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_768` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.6986 | | cosine_accuracy@3 | 0.83 | | cosine_accuracy@5 | 0.88 | | cosine_accuracy@10 | 0.9257 | | cosine_precision@1 | 0.6986 | | cosine_precision@3 | 0.2767 | | cosine_precision@5 | 0.176 | | cosine_precision@10 | 0.0926 | | cosine_recall@1 | 0.6986 | | cosine_recall@3 | 0.83 | | cosine_recall@5 | 0.88 | | cosine_recall@10 | 0.9257 | | cosine_ndcg@10 | 0.8142 | | cosine_mrr@10 | 0.7782 | | **cosine_map@100** | **0.7808** | #### Information Retrieval * Dataset: `dim_512` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7014 | | cosine_accuracy@3 | 0.8329 | | cosine_accuracy@5 | 0.8857 | | cosine_accuracy@10 | 0.9229 | | cosine_precision@1 | 0.7014 | | cosine_precision@3 | 0.2776 | | cosine_precision@5 | 0.1771 | | cosine_precision@10 | 0.0923 | | cosine_recall@1 | 0.7014 | | cosine_recall@3 | 0.8329 | | cosine_recall@5 | 0.8857 | | cosine_recall@10 | 0.9229 | | cosine_ndcg@10 | 0.8134 | | cosine_mrr@10 | 0.7781 | | **cosine_map@100** | **0.7809** | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7 | | cosine_accuracy@3 | 0.84 | | cosine_accuracy@5 | 0.8714 | | cosine_accuracy@10 | 0.9086 | | cosine_precision@1 | 0.7 | | cosine_precision@3 | 0.28 | | cosine_precision@5 | 0.1743 | | cosine_precision@10 | 0.0909 | | cosine_recall@1 | 0.7 | | cosine_recall@3 | 0.84 | | cosine_recall@5 | 0.8714 | | cosine_recall@10 | 0.9086 | | cosine_ndcg@10 | 0.8077 | | cosine_mrr@10 | 0.775 | | **cosine_map@100** | **0.7785** | #### Information Retrieval * Dataset: `dim_128` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.6943 | | cosine_accuracy@3 | 0.82 | | cosine_accuracy@5 | 0.8557 | | cosine_accuracy@10 | 0.9029 | | cosine_precision@1 | 0.6943 | | cosine_precision@3 | 0.2733 | | cosine_precision@5 | 0.1711 | | cosine_precision@10 | 0.0903 | | cosine_recall@1 | 0.6943 | | cosine_recall@3 | 0.82 | | cosine_recall@5 | 0.8557 | | cosine_recall@10 | 0.9029 | | cosine_ndcg@10 | 0.7991 | | cosine_mrr@10 | 0.7659 | | **cosine_map@100** | **0.7697** | #### Information Retrieval * Dataset: `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.6614 | | cosine_accuracy@3 | 0.7843 | | cosine_accuracy@5 | 0.8271 | | cosine_accuracy@10 | 0.8886 | | cosine_precision@1 | 0.6614 | | cosine_precision@3 | 0.2614 | | cosine_precision@5 | 0.1654 | | cosine_precision@10 | 0.0889 | | cosine_recall@1 | 0.6614 | | cosine_recall@3 | 0.7843 | | cosine_recall@5 | 0.8271 | | cosine_recall@10 | 0.8886 | | cosine_ndcg@10 | 0.7731 | | cosine_mrr@10 | 0.7366 | | **cosine_map@100** | **0.7404** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 6,300 training samples * Columns: positive and anchor * Approximate statistics based on the first 1000 samples: | | positive | anchor | |:--------|:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | positive | anchor | |:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------| | The Centers for Medicare & Medicaid Services issued a final rule in October 2023 for the calendar year 2024, estimating a productivity-adjusted market basket increase of 2.1% in average reimbursement to ESRD facilities. | What is the projected impact on average reimbursement to ESRD facilities in 2024 due to the final rule issued by CMS? | | Company Adjusted EBIT Margin is derived by dividing the Company adjusted EBIT by Company revenue, which is a non-GAAP measure useful for evaluating the company's operating results. | How is the Company Adjusted EBIT Margin calculated? | | The provision for credit losses was $4 million for the year ended December 31, 202 serviLists of account holders responsible for and the state of the economy, our credit standards, our risk assessments, and the judgment of our employees responsible for granting credit. | What factors influence the provision for credit losses at Las Vegas Sands Corp.? | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 768, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `bf16`: True - `tf32`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `torch_empty_cache_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: True - `fp16`: False - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: True - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `eval_on_start`: False - `eval_use_gather_object`: False - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 | |:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|:----------------------:| | 0.8122 | 10 | 1.5176 | - | - | - | - | - | | 0.9746 | 12 | - | 0.7500 | 0.7642 | 0.7680 | 0.7079 | 0.7708 | | 1.6244 | 20 | 0.6868 | - | - | - | - | - | | 1.9492 | 24 | - | 0.7657 | 0.7746 | 0.7784 | 0.7323 | 0.7816 | | 2.4365 | 30 | 0.4738 | - | - | - | - | - | | 2.9239 | 36 | - | 0.7691 | 0.7780 | 0.7790 | 0.7402 | 0.7796 | | 3.2487 | 40 | 0.3934 | - | - | - | - | - | | **3.8985** | **48** | **-** | **0.7697** | **0.7785** | **0.7809** | **0.7404** | **0.7808** | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.11.9 - Sentence Transformers: 3.0.1 - Transformers: 4.44.2 - PyTorch: 2.4.0+cu121 - Accelerate: 0.33.0 - Datasets: 2.21.0 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```