README.md

metadata

base_model: BAAI/bge-m3
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:828
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      Comunicació prèvia per l'execució de cales, pous i sondejos, en terreny
      privat, previs a l'actuació definitiva.
    sentences:
      - Quin és el requisit per a l'execució de les obres en terreny privat?
      - Quin és el propòsit del tràmit de rectificació de dades personals?
      - Quin és el requisit per a la crema en zones de conservació?
  - source_sentence: >-
      En el mateix tràmit també es pot actualitzar el canvi de domicili o dades
      personals, si escau.
    sentences:
      - Quins tributs puc domiciliar amb aquest tràmit?
      - >-
        Quin és el compromís del titular de l'activitat en la Declaració
        responsable?
      - Quin és el tràmit que permet actualitzar les dades personals?
  - source_sentence: >-
      El reconeixement administratiu del dret comunicat es produeix salvat el
      dret de propietat, sens perjudici del de tercers ni de les competències
      d’altres organismes i administracions.
    sentences:
      - >-
        Quin és el tràmit que permet una major transparència en la gestió dels
        animals domèstics?
      - Quin és el requisit per considerar una tala de masses arbòries?
      - Quin és el reconeixement administratiu del dret comunicat?
  - source_sentence: >-
      El seu objecte és que -prèviament a la seva execució material-
      l'Ajuntament comprovi l'adequació de l’actuació a la normativa i
      planejament, així com a les ordenances municipals.
    sentences:
      - Quin és el resultat de rectificar les meves dades personals?
      - >-
        Quin és el paper de les llicències urbanístiques en la instal·lació de
        construccions auxiliars o mòduls prefabricats?
      - Quin és l'objectiu de l'Ajuntament en aquest tràmit?
  - source_sentence: >-
      Permet sol·licitar l’autorització per a l’ús comú especial de la via
      pública per a reserves temporals d’estacionament i espai públic per:
      càrrega/descàrrega de materials diversos davant d'una obra;
    sentences:
      - >-
        Quin és el propòsit de les actuacions de manteniment d'elements de
        façana i cobertes?
      - >-
        Quin és el tràmit per canviar el domicili del permís de conducció i del
        permís de circulació?
      - >-
        Quins són els materials que es poden càrregar/descarregar en l'ocupació
        i reserves temporals amb càrrega/descàrrega de materials?
model-index:
  - name: SentenceTransformer based on BAAI/bge-m3
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 1024
          type: dim_1024
        metrics:
          - type: cosine_accuracy@1
            value: 0.1956521739130435
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5434782608695652
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6739130434782609
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7717391304347826
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.1956521739130435
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18115942028985504
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13478260869565215
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07717391304347823
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.1956521739130435
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5434782608695652
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6739130434782609
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7717391304347826
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.48504415203944085
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.39229641131815035
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4002530280745044
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.1956521739130435
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5543478260869565
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6739130434782609
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7717391304347826
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.1956521739130435
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18478260869565213
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13478260869565215
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07717391304347823
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.1956521739130435
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5543478260869565
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6739130434782609
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7717391304347826
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.48804421462232656
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3962215320910973
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.404212372178018
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.20652173913043478
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5434782608695652
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6521739130434783
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7608695652173914
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.20652173913043478
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18115942028985504
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13043478260869562
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07608695652173911
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.20652173913043478
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5434782608695652
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6521739130434783
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7608695652173914
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4840641874049137
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.39500086266390616
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4031258766496075
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.18478260869565216
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5434782608695652
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6521739130434783
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.75
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.18478260869565216
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18115942028985504
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13043478260869562
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07499999999999998
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.18478260869565216
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5434782608695652
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6521739130434783
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.75
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4702420475154915
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.3799301242236025
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.38860307402910876
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.22826086956521738
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5434782608695652
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6956521739130435
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.782608695652174
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.22826086956521738
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.18115942028985504
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.13913043478260867
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07826086956521737
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.22826086956521738
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.5434782608695652
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6956521739130435
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.782608695652174
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5045819494113778
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.41489820565907526
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4206777643300118
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.17391304347826086
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.4891304347826087
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.6630434782608695
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.7608695652173914
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.17391304347826086
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16304347826086954
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.1326086956521739
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.07608695652173911
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.17391304347826086
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.4891304347826087
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.6630434782608695
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.7608695652173914
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4628441336923734
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.36670548654244295
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.37290616382203134
            name: Cosine Map@100

SentenceTransformer based on BAAI/bge-m3

This is a sentence-transformers model finetuned from BAAI/bge-m3 on the json dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-m3
• Maximum Sequence Length: 8192 tokens
• Output Dimensionality: 1024 tokens
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("adriansanz/sqv-v3")
# Run inference
sentences = [
    "Permet sol·licitar l’autorització per a l’ús comú especial de la via pública per a reserves temporals d’estacionament i espai públic per: càrrega/descàrrega de materials diversos davant d'una obra;",
    "Quins són els materials que es poden càrregar/descarregar en l'ocupació i reserves temporals amb càrrega/descàrrega de materials?",
    'Quin és el tràmit per canviar el domicili del permís de conducció i del permís de circulació?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

• Dataset: dim_1024
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1957
cosine_accuracy@3	0.5435
cosine_accuracy@5	0.6739
cosine_accuracy@10	0.7717
cosine_precision@1	0.1957
cosine_precision@3	0.1812
cosine_precision@5	0.1348
cosine_precision@10	0.0772
cosine_recall@1	0.1957
cosine_recall@3	0.5435
cosine_recall@5	0.6739
cosine_recall@10	0.7717
cosine_ndcg@10	0.485
cosine_mrr@10	0.3923
cosine_map@100	0.4003

Information Retrieval

• Dataset: dim_768
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1957
cosine_accuracy@3	0.5543
cosine_accuracy@5	0.6739
cosine_accuracy@10	0.7717
cosine_precision@1	0.1957
cosine_precision@3	0.1848
cosine_precision@5	0.1348
cosine_precision@10	0.0772
cosine_recall@1	0.1957
cosine_recall@3	0.5543
cosine_recall@5	0.6739
cosine_recall@10	0.7717
cosine_ndcg@10	0.488
cosine_mrr@10	0.3962
cosine_map@100	0.4042

Information Retrieval

• Dataset: dim_512
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2065
cosine_accuracy@3	0.5435
cosine_accuracy@5	0.6522
cosine_accuracy@10	0.7609
cosine_precision@1	0.2065
cosine_precision@3	0.1812
cosine_precision@5	0.1304
cosine_precision@10	0.0761
cosine_recall@1	0.2065
cosine_recall@3	0.5435
cosine_recall@5	0.6522
cosine_recall@10	0.7609
cosine_ndcg@10	0.4841
cosine_mrr@10	0.395
cosine_map@100	0.4031

Information Retrieval

• Dataset: dim_256
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1848
cosine_accuracy@3	0.5435
cosine_accuracy@5	0.6522
cosine_accuracy@10	0.75
cosine_precision@1	0.1848
cosine_precision@3	0.1812
cosine_precision@5	0.1304
cosine_precision@10	0.075
cosine_recall@1	0.1848
cosine_recall@3	0.5435
cosine_recall@5	0.6522
cosine_recall@10	0.75
cosine_ndcg@10	0.4702
cosine_mrr@10	0.3799
cosine_map@100	0.3886

Information Retrieval

• Dataset: dim_128
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.2283
cosine_accuracy@3	0.5435
cosine_accuracy@5	0.6957
cosine_accuracy@10	0.7826
cosine_precision@1	0.2283
cosine_precision@3	0.1812
cosine_precision@5	0.1391
cosine_precision@10	0.0783
cosine_recall@1	0.2283
cosine_recall@3	0.5435
cosine_recall@5	0.6957
cosine_recall@10	0.7826
cosine_ndcg@10	0.5046
cosine_mrr@10	0.4149
cosine_map@100	0.4207

Information Retrieval

• Dataset: dim_64
• Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1739
cosine_accuracy@3	0.4891
cosine_accuracy@5	0.663
cosine_accuracy@10	0.7609
cosine_precision@1	0.1739
cosine_precision@3	0.163
cosine_precision@5	0.1326
cosine_precision@10	0.0761
cosine_recall@1	0.1739
cosine_recall@3	0.4891
cosine_recall@5	0.663
cosine_recall@10	0.7609
cosine_ndcg@10	0.4628
cosine_mrr@10	0.3667
cosine_map@100	0.3729

Training Details

Training Dataset

json

• Dataset: json
• Size: 828 training samples
• Columns: positive and anchor
• Approximate statistics based on the first 828 samples:

  	positive	anchor
  type	string	string
  details	min: 5 tokens mean: 41.95 tokens max: 117 tokens	min: 9 tokens mean: 20.81 tokens max: 50 tokens

• Samples:

  positive	anchor
  Consultar l'estat tributari d'un contribuent. Us permet consultar l'estat dels rebuts i liquidacions que estan a nom del contribuent titular d'un certificat electrònic, així com els elements que configuren el càlcul per determinar el deute tributari de cadascun d'ells.	Com puc consultar l'estat tributari d'un contribuent?
  L'informe facultatiu servirà per tramitar una autorització de residència temporal per arrelament social.	Quin és el tràmit relacionat amb la residència a l'Ajuntament?
  Aquesta targeta, és el document que dona dret a persones físiques o jurídiques titulars de vehicles adaptats destinats al transport col·lectiu de persones amb discapacitat...	Quin és el benefici de tenir la targeta d'aparcament de transport col·lectiu per a les persones amb discapacitat?

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.2
• bf16: True
• tf32: True
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: epoch
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 16
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 5
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.2
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: True
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: False
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• eval_use_gather_object: False
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_1024_cosine_map@100	dim_128_cosine_map@100	dim_256_cosine_map@100	dim_512_cosine_map@100	dim_64_cosine_map@100	dim_768_cosine_map@100
0.9231	3	-	0.3914	0.3466	0.3625	0.3778	0.3067	0.3810
1.8462	6	-	0.3835	0.3940	0.3789	0.3857	0.3407	0.3808
2.7692	9	-	0.4028	0.4159	0.3961	0.4098	0.3803	0.4029
3.0769	10	3.1546	-	-	-	-	-	-
4.0	13	-	0.3992	0.4209	0.3905	0.4121	0.3806	0.4009
4.6154	15	-	0.4003	0.4207	0.3886	0.4031	0.3729	0.4042

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.1.1
• Transformers: 4.44.2
• PyTorch: 2.4.1+cu121
• Accelerate: 0.35.0.dev0
• Datasets: 3.0.1
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}