metadata

base_model: BAAI/bge-base-en-v1.5
datasets: []
language:
  - en
library_name: sentence-transformers
license: apache-2.0
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_ndcg@100
  - cosine_mrr@10
  - cosine_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:6201
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: ' entirety. This is a form of ownership that can only be created by married persons. Both spouses hold title to the whole property with the right of survivorship. When one spouse dies, the surviving spouse takes title to the property. When the second spouse dies, the property is distributed to the heirs according to the terms of the will. Tenants in Common. Jointly owned assets may also be held as tenants in common. With this form of ownership, each owner  holds a share of the property, which may or may not be equal. When one owner dies, his or her share passes immediately to that persons heirs, according to the laws in each state. Bank accounts, securities accounts and certificates of deposit can be set up as joint accounts, which may provide liquidity after your death. For example, you could open a joint checking account, with right of YOUR LEGACY An Estate-Planning Guide 13 survivorship, with one of your adult children. After your death, the adult child would'
    sentences:
      - What determines the date of deposit?
      - >-
        What are the advantages of shopping online and how can you find and
        compare products easily?
      - >-
        What are the different forms of ownership in real estate and how do they
        work?
  - source_sentence: ' If you''re starting the new year with credit card debt, focus on creating a plan for bringing the balances down. And remember to track your progress so you have a motivational boost to stick with it. Why is a Good Credit Score Important? A good credit score can open a variety of financial doors. Higher credit scores can allow you to qualify for premium credit cards with better rewards and perks. An excellent credit score can also help you qualify for certain loans and mortgages, or even get better interest rates on the loans that you qualify for. With poor or no credit history, many financial products may be unavailable. But if you start implementing these keyways to improve your credit score, youll be on track to a better credit score and all the benefits that come with it. Using a Citi Secured Mastercard If youre just starting your credit journey, it may be hard to see what credit products you can qualify for. A secured credit card like the Citi Secured Mastercard is a great entry'
    sentences:
      - What are the benefits of having a good credit score?
      - What is the purpose of the above information provided by Citi?
      - When is the Best Time to Apply for a Credit Card?
  - source_sentence: ' decreased rate of return on the reinvestment of the proceeds received as a result of a payment on a Deposit prior to its scheduled maturity,  payment in cash of the Deposit principal prior to maturity in connection with the liquidation of an insured institution or the assumption of all or a portion of its deposit liabilities at a lower interest rate or  its 29 receipt of a decreased rate of return as compared to the return on the applicable securities, indices, currencies, intangibles, articles, commodities or goods or any other economic measure or instrument, including the occurrence or non-occurrence of any event. Preference in Right of Payment Federal legislation adopted in 1993 provides for a preference in right of payment of certain claims made in the liquidation or other resolution of any FDIC-insured depository institution. The statute requires claims to be paid in the following order: First, administrative expenses of the receiver; Second, any deposit liability of the institution; Third, any other general or senior liability of the'
    sentences:
      - >-
        How can I protect myself from fake Citi SMS texts and fraudulent money
        transfers?
      - >-
        What are the details required to transfer funds out of my account and
        what are the different types of payments available for transferring
        funds out of my account?
      - >-
        What is the mechanism for decreased rate of return on reinvestment of
        the proceeds received as a result of a payment on a Deposit prior to its
        scheduled maturity?
  - source_sentence: ' Citigroup Inc. All rights reserved. Citi, Citi and Arc Design and other marks used herein are service marks of Citigroup Inc. or its affliates, used and registered throughout the world. 2164316 GTS26358 0223 Tips to Become a Smart Credit Card User Citi.com - ATM Branch - Open an Account - Espaol !Citibank LogoSearch!Search Citi.com Menu - Credit Cards - View All Credit Cards - 0 Intro APR Credit Cards - Balance Transfer Credit Cards - Cash Back Credit Cards - Rewards Credit Cards - See If You''re Pre-Selected - Small Business Credit Cards - Banking - Banking Overview - Checking - Savings - Certificates of Deposit - Banking IRAs - Rates - Small Business Banking - Lending - Personal Loans Lines of Credit - Mortgage - Home Equity - Small Business Lending - Investing - Investing with Citi - Self Directed Trading - Citigold - Credit Cards - Credit Knowledge Center - Understanding Credit Cards - Tips'
    sentences:
      - What are the tips to become a smart credit card user?
      - >-
        What information do we request and receive from you to explain
        transactions or attempted transactions in or through your account?
      - >-
        Who has permission from the primary cardholder to use the credit card
        account and receive their own card with their own name?
  - source_sentence: ' and Arc Design is a registered service mark of Citigroup Inc. OpenInvestor is a service mark of Citigroup Inc. 1044398 GTS74053 0113 Trade Working Capital Viewpoints Navigating global uncertainty: Perspectives on supporting the healthcare supply chain November 2023 Treasury and Trade Solutions Foreword Foreword Since the inception of the COVID-19 pandemic, the healthcare industry has faced supply chain disruptions. The industry, which has a long tradition in innovation, continues to transform to meet the needs of an evolving environment. Pauline kXXXXX Unlocking the full potential within the healthcare industry Global Head, Trade requires continuous investment. As corporates plan for the Working Capital Advisory future, careful working capital management is essential to ensuring they get there. Andrew Betts Global head of TTS Trade Sales Client Management, Citi Bayo Gbowu Global Sector Lead, Trade Healthcare and Wellness Ian Kervick-Jimenez Trade Working Capital Advisory 2 Treasury and Trade Solutions The Working'
    sentences:
      - >-
        How can I manage my Citibank accounts through International Personal
        Bank U.S., either via internet, text messages, or phone calls?
      - What are the registered service marks of Citigroup Inc?
      - >-
        What is the role of DXX jXXXX US Real Estate Total Return SM Index in
        determining, composing or calculating products?
model-index:
  - name: SentenceTransformer based on BAAI/bge-base-en-v1.5
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.49420289855072463
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.6768115942028986
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.7478260869565218
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.8333333333333334
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.49420289855072463
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.22560386473429955
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.14956521739130432
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.08333333333333333
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.49420289855072463
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.6768115942028986
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.7478260869565218
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.8333333333333334
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.6585419708540992
            name: Cosine Ndcg@10
          - type: cosine_ndcg@100
            value: 0.6900535995185644
            name: Cosine Ndcg@100
          - type: cosine_mrr@10
            value: 0.6032240625718881
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6096261483024806
            name: Cosine Map@100

SentenceTransformer based on BAAI/bge-base-en-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-en-v1.5
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("MugheesAwan11/bge-base-citi-dataset-detailed-6k-0_5k-e2")
# Run inference
sentences = [
    ' and Arc Design is a registered service mark of Citigroup Inc. OpenInvestor is a service mark of Citigroup Inc. 1044398 GTS74053 0113 Trade Working Capital Viewpoints Navigating global uncertainty: Perspectives on supporting the healthcare supply chain November 2023 Treasury and Trade Solutions Foreword Foreword Since the inception of the COVID-19 pandemic, the healthcare industry has faced supply chain disruptions. The industry, which has a long tradition in innovation, continues to transform to meet the needs of an evolving environment. Pauline kXXXXX Unlocking the full potential within the healthcare industry Global Head, Trade requires continuous investment. As corporates plan for the Working Capital Advisory future, careful working capital management is essential to ensuring they get there. Andrew Betts Global head of TTS Trade Sales Client Management, Citi Bayo Gbowu Global Sector Lead, Trade Healthcare and Wellness Ian Kervick-Jimenez Trade Working Capital Advisory 2 Treasury and Trade Solutions The Working',
    'What are the registered service marks of Citigroup Inc?',
    'What is the role of DXX jXXXX US Real Estate Total Return SM Index in determining, composing or calculating products?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.4942
cosine_accuracy@3	0.6768
cosine_accuracy@5	0.7478
cosine_accuracy@10	0.8333
cosine_precision@1	0.4942
cosine_precision@3	0.2256
cosine_precision@5	0.1496
cosine_precision@10	0.0833
cosine_recall@1	0.4942
cosine_recall@3	0.6768
cosine_recall@5	0.7478
cosine_recall@10	0.8333
cosine_ndcg@10	0.6585
cosine_ndcg@100	0.6901
cosine_mrr@10	0.6032
cosine_map@100	0.6096

Training Details

Training Dataset

Unnamed Dataset

Size: 6,201 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 146 tokens
mean: 205.96 tokens
max: 289 tokens

min: 8 tokens
mean: 26.75 tokens
max: 241 tokens

	positive	anchor
type	string	string
details	min: 146 tokens mean: 205.96 tokens max: 289 tokens	min: 8 tokens mean: 26.75 tokens max: 241 tokens

Samples:

positive	anchor
combined balances do not include: balances in delinquent accounts; balances that exceed your approved credit When Deposits Are Credited to an Account limit for any line of credit or credit card; or outstanding balances Deposits received before the end of a Business Day will be credited to your account that day. However, there been established for the Citigold Account Package. Your may be a delay before these funds are available for your use. See combined monthly balance range will be determined by computing the Funds Availability at Citibank section of this Marketplace an average of your monthly balances for your linked accounts Addendum for more information. during the prior calendar month. Monthly service fees are applied only to accounts with a combined average monthly balance range under the specified limits starting two statement cycles after account opening. Service fees assessed will appear as a charge on your next statement. 2 3 Combined Average Monthly Non- Per Special Circumstances Monthly Balance Service Citibank Check If a checking account is converted	`What are the conditions for balances to be included in the combined balances?`
the first six months, your credit score may not be where you want it just yet. There are other factors that impact your credit score including the length of your credit file, your credit mix and your credit utilization. If youre hoping to repair a credit score that has been damaged by financial setbacks, the timelines can be longer. A year or two with regular, timely payments and good credit utilization can push your credit score up. However, bankruptcies, collections, and late payments can linger on your credit report for anywhere from seven to ten years. That said, you may not have to use a secured credit card throughout your entire credit building process. Your goal may be to repair your credit to the point where your credit score is good enough to make you eligible for an unsecured credit card. To that end, youll need to research factors such as any fees that apply to the unsecured credit cards youre considering. There is no quick fix to having a great credit score. Building good credit with a	`What factors impact your credit score including the length of your credit file, your credit mix, and your credit utilization?`
by the index sponsor of the Constituents when it calculated the hypothetical back-tested index levels for the Constituents. It is impossible to predict whether the Index will rise or fall. The actual future performance of the Index may bear no relation to the historical or hypothetical back-tested levels of the Index. The Index Administrator, which is our Affiliate, and the Index Calculation Agent May Exercise Judgments under Certain Circumstances in the Calculation of the Index. Although the Index is rules- based, there are certain circumstances under which the Index Administrator or Index Calculation Agent may be required to exercise judgment in calculating the Index, including the following: The Index Administrator will determine whether an ambiguity, error or omission has arisen and the Index Administrator may resolve such ambiguity, error or omission, acting in good faith and in a commercially reasonable manner, and may amend the Index Rules to reflect the resolution of the ambiguity, error or omission in a manner that is consistent with the commercial objective of the Index. The Index Calculation Agents calculations	`What circumstances may require the Index Administrator or Index Calculation Agent to exercise judgment in calculating the Index?`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768
    ],
    "matryoshka_weights": [
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
learning_rate: 2e-05
num_train_epochs: 2
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: True
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_768_cosine_map@100
0.0515	10	0.7623	-
0.1031	20	0.6475	-
0.1546	30	0.4492	-
0.2062	40	0.3238	-
0.2577	50	0.2331	-
0.3093	60	0.2575	-
0.3608	70	0.3619	-
0.4124	80	0.1539	-
0.4639	90	0.1937	-
0.5155	100	0.241	-
0.5670	110	0.2192	-
0.6186	120	0.2553	-
0.6701	130	0.2438	-
0.7216	140	0.1916	-
0.7732	150	0.189	-
0.8247	160	0.1721	-
0.8763	170	0.2353	-
0.9278	180	0.1713	-
0.9794	190	0.2121	-
1.0	194	-	0.6100
1.0309	200	0.1394	-
1.0825	210	0.156	-
1.1340	220	0.1276	-
1.1856	230	0.0969	-
1.2371	240	0.0811	-
1.2887	250	0.0699	-
1.3402	260	0.0924	-
1.3918	270	0.0838	-
1.4433	280	0.064	-
1.4948	290	0.0624	-
1.5464	300	0.0837	-
1.5979	310	0.0881	-
1.6495	320	0.1065	-
1.7010	330	0.0646	-
1.7526	340	0.084	-
1.8041	350	0.0697	-
1.8557	360	0.0888	-
1.9072	370	0.0873	-
1.9588	380	0.0755	-
2.0	388	-	0.6096

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.14
Sentence Transformers: 3.0.1
Transformers: 4.41.2
PyTorch: 2.1.2+cu121
Accelerate: 0.32.1
Datasets: 2.19.1
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning}, 
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}