metadata

base_model: sentence-transformers/all-MiniLM-L6-v2
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
  - dot_accuracy@1
  - dot_accuracy@3
  - dot_accuracy@5
  - dot_accuracy@10
  - dot_precision@1
  - dot_precision@3
  - dot_precision@5
  - dot_precision@10
  - dot_recall@1
  - dot_recall@3
  - dot_recall@5
  - dot_recall@10
  - dot_ndcg@10
  - dot_mrr@10
  - dot_map@100
pipeline_tag: sentence-similarity
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:760
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
widget:
  - source_sentence: >-
      Why is it important to establish clear timelines for data retention, and
      what should happen to data once those timelines are reached?
    sentences:
      - |-
        Technology 
        Dignari 
        Douglas Goddard 
        Edgar Dworsky 
        Electronic Frontier Foundation 
        Electronic Privacy Information 
        Center, Center for Digital 
        Democracy, and Consumer 
        Federation of America 
        FaceTec 
        Fight for the Future 
        Ganesh Mani 
        Georgia Tech Research Institute 
        Google 
        Health Information Technology 
        Research and Development 
        Interagency Working Group 
        HireVue 
        HR Policy Association 
        ID.me 
        Identity and Data Sciences 
        Laboratory at Science Applications 
        International Corporation 
        Information Technology and 
        Innovation Foundation 
        Information Technology Industry 
        Council 
        Innocence Project 
        Institute for Human-Centered 
        Artificial Intelligence at Stanford 
        University 
        Integrated Justice Information 
        Systems Institute 
        International Association of Chiefs 
        of Police 
        International Biometrics + Identity 
        Association 
        International Business Machines 
        Corporation 
        International Committee of the Red 
        Cross 
        Inventionphysics 
        iProov 
        Jacob Boudreau 
        Jennifer K. Wagner, Dan Berger,
      - >-
        new privacy risks and implementing appropriate mitigation measures,
        which may include express consent. 

        Clear timelines for data retention should be established, with data
        deleted as soon as possible in accordance 

        with legal or policy-based limitations. Determined data retention
        timelines should be documented and justi

        fied. 

        Risk identification and mitigation. Entities that collect, use, share,
        or store sensitive data should 

        attempt to proactively identify harms and seek to manage them so as to
        avoid, mitigate, and respond appropri

        ately to identified risks. Appropriate responses include determining not
        to process data when the privacy risks 

        outweigh the benefits or implementing measures to mitigate acceptable
        risks. Appropriate responses do not 

        include sharing or transferring the privacy risks to users via notice or
        consent requests where users could not 

        reasonably be expected to understand the risks without further support.
      - >-
        55. Data & Trust Alliance. Algorithmic Bias Safeguards for Workforce:
        Overview. Jan. 2022. https://

        dataandtrustalliance.org/Algorithmic_Bias_Safeguards_for_Workforce_Overview.pdf

        56. Section 508.gov. IT Accessibility Laws and Policies. Access Board.
        https://www.section508.gov/

        manage/laws-and-policies/

        67
  - source_sentence: What is the purpose of the NIST AI Risk Management Framework?
    sentences:
      - |-
        TABLE OF CONTENTS
        FROM PRINCIPLES TO PRACTICE: A TECHNICAL COMPANION TO THE BLUEPRINT 
        FOR AN AI BILL OF RIGHTS 
         
        USING THIS TECHNICAL COMPANION
         
        SAFE AND EFFECTIVE SYSTEMS
         
        ALGORITHMIC DISCRIMINATION PROTECTIONS
         
        DATA PRIVACY
         
        NOTICE AND EXPLANATION
         
        HUMAN ALTERNATIVES, CONSIDERATION, AND FALLBACK
        APPENDIX
         
        EXAMPLES OF AUTOMATED SYSTEMS
         
        LISTENING TO THE AMERICAN PEOPLE
        ENDNOTES 
        12
        14
        15
        23
        30
        40
        46
        53
        53
        55
        63
        13
      - >-
        health diagnostic systems. 

        The Blueprint for an AI Bill of Rights recognizes that law enforcement
        activities require a balancing of 

        equities, for example, between the protection of sensitive law
        enforcement information and the principle of 

        notice; as such, notice may not be appropriate, or may need to be
        adjusted to protect sources, methods, and 

        other law enforcement equities. Even in contexts where these principles
        may not apply in whole or in part, 

        federal departments and agencies remain subject to judicial, privacy,
        and civil liberties oversight as well as 

        existing policies and safeguards that govern automated systems,
        including, for example, Executive Order 13960, 

        Promoting the Use of Trustworthy Artificial Intelligence in the Federal
        Government (December 2020). 

        This white paper recognizes that national security (which includes
        certain law enforcement and 

        homeland security activities) and defense activities are of increased
        sensitivity and interest to our nation’s
      - >-
        mitigate risks posed by the use of AI to companies’ reputation, legal
        responsibilities, and other product safety 

        and effectiveness concerns. 

        The Office of Management and Budget (OMB) has called for an expansion of
        opportunities 

        for meaningful stakeholder engagement in the design of programs and
        services. OMB also 

        points to numerous examples of effective and proactive stakeholder
        engagement, including the Community-

        Based Participatory Research Program developed by the National
        Institutes of Health and the participatory 

        technology assessments developed by the National Oceanic and Atmospheric
        Administration.18

        The National Institute of Standards and Technology (NIST) is developing
        a risk 

        management framework to better manage risks posed to individuals,
        organizations, and 

        society by AI.19 The NIST AI Risk Management Framework, as mandated by
        Congress, is intended for 

        voluntary use to help incorporate trustworthiness considerations into
        the design, development, use, and
  - source_sentence: >-
      What were the main topics discussed in the panel focused on consumer
      rights and protections in an automated society?
    sentences:
      - >-
        context, or may be more speculative and therefore uncertain. 

        AI risks can diﬀer from or intensify traditional software risks.
        Likewise, GAI can exacerbate existing AI 

        risks, and creates unique risks. GAI risks can vary along many
        dimensions: 

        • 

        Stage of the AI lifecycle: Risks can arise during design, development,
        deployment, operation, 

        and/or decommissioning. 

        • 

        Scope: Risks may exist at individual model or system levels, at the
        application or implementation 

        levels (i.e., for a speciﬁc use case), or at the ecosystem level – that
        is, beyond a single system or 

        organizational context. Examples of the latter include the expansion of
        “algorithmic 

        monocultures,3” resulting from repeated use of the same model, or
        impacts on access to 

        opportunity, labor markets, and the creative economies.4 

        • 

        Source of risk: Risks may emerge from factors related to the design,
        training, or operation of the
      - >-
        speciﬁc and empirically well-substantiated negative risk to public
        safety (or has 

        already caused harm). 

        CBRN Information or Capabilities; 

        Dangerous, Violent, or Hateful 

        Content 

        AI Actor Tasks: Governance and Oversight
      - >-
        theme, exploring current challenges and concerns and considering what an
        automated society that 

        respects democratic values should look like. These discussions focused
        on the topics of consumer 

        rights and protections, the criminal justice system, equal opportunities
        and civil justice, artificial 

        intelligence and democratic values, social welfare and development, and
        the healthcare system. 

        Summaries of Panel Discussions: 

        Panel 1: Consumer Rights and Protections. This event explored the
        opportunities and challenges for 

        individual consumers and communities in the context of a growing
        ecosystem of AI-enabled consumer 

        products, advanced platforms and services, “Internet of Things” (IoT)
        devices, and smart city products and 

        services. 

        Welcome:

        •

        Rashida Richardson, Senior Policy Advisor for Data and Democracy, White
        House Office of Science and

        Technology Policy

        •

        Karen Kornbluh, Senior Fellow and Director of the Digital Innovation and
        Democracy Initiative, German

        Marshall Fund
  - source_sentence: >-
      How did the input from various stakeholders contribute to the development
      of the Blueprint for an AI Bill of Rights?
    sentences:
      - >-
        SECTION TITLE

        APPENDIX

        Listening to the American People 

        The White House Office of Science and Technology Policy (OSTP) led a
        yearlong process to seek and distill 

        input from people across the country – from impacted communities to
        industry stakeholders to 

        technology developers to other experts across fields and sectors, as
        well as policymakers across the Federal 

        government – on the issue of algorithmic and data-driven harms and
        potential remedies. Through panel 

        discussions, public listening sessions, private meetings, a formal
        request for information, and input to a 

        publicly accessible and widely-publicized email address, people across
        the United States spoke up about 

        both the promises and potential harms of these technologies, and played
        a central role in shaping the 

        Blueprint for an AI Bill of Rights. 

        Panel Discussions to Inform the Blueprint for An AI Bill of Rights
      - >-
        About this Document 

        The Blueprint for an AI Bill of Rights: Making Automated Systems Work
        for the American People was 

        published by the White House Office of Science and Technology Policy in
        October 2022. This framework was 

        released one year after OSTP announced the launch of a process to
        develop “a bill of rights for an AI-powered 

        world.” Its release follows a year of public engagement to inform this
        initiative. The framework is available 

        online at: https://www.whitehouse.gov/ostp/ai-bill-of-rights 

        About the Office of Science and Technology Policy 

        The Office of Science and Technology Policy (OSTP) was established by
        the National Science and Technology 

        Policy, Organization, and Priorities Act of 1976 to provide the
        President and others within the Executive Office 

        of the President with advice on the scientific, engineering, and
        technological aspects of the economy, national
      - >-
        Technology Policy

        •

        Karen Kornbluh, Senior Fellow and Director of the Digital Innovation and
        Democracy Initiative, German

        Marshall Fund

        Moderator: 

        Devin E. Willis, Attorney, Division of Privacy and Identity Protection,
        Bureau of Consumer Protection, Federal 

        Trade Commission 

        Panelists: 

        •

        Tamika L. Butler, Principal, Tamika L. Butler Consulting

        •

        Jennifer Clark, Professor and Head of City and Regional Planning,
        Knowlton School of Engineering, Ohio

        State University

        •

        Carl Holshouser, Senior Vice President for Operations and Strategic
        Initiatives, TechNet

        •

        Surya Mattu, Senior Data Engineer and Investigative Data Journalist, The
        Markup

        •

        Mariah Montgomery, National Campaign Director, Partnership for Working
        Families

        55
  - source_sentence: >-
      What legal action did the Federal Trade Commission take against Kochava
      regarding data tracking?
    sentences:
      - >-
        DATA PRIVACY 

        EXTRA PROTECTIONS FOR DATA RELATED TO SENSITIVE

        DOMAINS

        •

        Continuous positive airway pressure machines gather data for medical
        purposes, such as diagnosing sleep

        apnea, and send usage data to a patient’s insurance company, which may
        subsequently deny coverage for the

        device based on usage data. Patients were not aware that the data would
        be used in this way or monitored

        by anyone other than their doctor.70 

        •

        A department store company used predictive analytics applied to
        collected consumer data to determine that a

        teenage girl was pregnant, and sent maternity clothing ads and other
        baby-related advertisements to her

        house, revealing to her father that she was pregnant.71

        •

        School audio surveillance systems monitor student conversations to
        detect potential "stress indicators" as

        a warning of potential violence.72 Online proctoring systems claim to
        detect if a student is cheating on an
      - >-
        ENDNOTES

        75. See., e.g., Sam Sabin. Digital surveillance in a post-Roe world.
        Politico. May 5, 2022. https://

        www.politico.com/newsletters/digital-future-daily/2022/05/05/digital-surveillance-in-a-post-roe

        world-00030459; Federal Trade Commission. FTC Sues Kochava for Selling
        Data that Tracks People at

        Reproductive Health Clinics, Places of Worship, and Other Sensitive
        Locations. Aug. 29, 2022. https://

        www.ftc.gov/news-events/news/press-releases/2022/08/ftc-sues-kochava-selling-data-tracks-people

        reproductive-health-clinics-places-worship-other

        76. Todd Feathers. This Private Equity Firm Is Amassing Companies That
        Collect Data on America’s

        Children. The Markup. Jan. 11, 2022.

        https://themarkup.org/machine-learning/2022/01/11/this-private-equity-firm-is-amassing-companies

        that-collect-data-on-americas-children

        77. Reed Albergotti. Every employee who leaves Apple becomes an
        ‘associate’: In job databases used by
      - "ENDNOTES\n1.The Executive Order On Advancing Racial Equity and Support for Underserved Communities Through the\nFederal\_Government. https://www.whitehouse.gov/briefing-room/presidential-actions/2021/01/20/executive\norder-advancing-racial-equity-and-support-for-underserved-communities-through-the-federal-government/\n2. The White House. Remarks by President Biden on the Supreme Court Decision to Overturn Roe v. Wade. Jun.\n24, 2022. https://www.whitehouse.gov/briefing-room/speeches-remarks/2022/06/24/remarks-by-president\nbiden-on-the-supreme-court-decision-to-overturn-roe-v-wade/\n3. The White House. Join the Effort to Create A Bill of Rights for an Automated Society. Nov. 10, 2021. https://\nwww.whitehouse.gov/ostp/news-updates/2021/11/10/join-the-effort-to-create-a-bill-of-rights-for-an\nautomated-society/\n4. U.S. Dept. of Health, Educ. & Welfare, Report of the Sec’y’s Advisory Comm. on Automated Pers. Data Sys.,"
model-index:
  - name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: Unknown
          type: unknown
        metrics:
          - type: cosine_accuracy@1
            value: 0.7214285714285714
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.8785714285714286
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.95
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.9714285714285714
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.7214285714285714
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.2928571428571428
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.18999999999999995
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.09714285714285713
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.7214285714285714
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.8785714285714286
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.95
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.9714285714285714
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.8514639427234363
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.8122108843537416
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.8142292826221397
            name: Cosine Map@100
          - type: dot_accuracy@1
            value: 0.7214285714285714
            name: Dot Accuracy@1
          - type: dot_accuracy@3
            value: 0.8785714285714286
            name: Dot Accuracy@3
          - type: dot_accuracy@5
            value: 0.95
            name: Dot Accuracy@5
          - type: dot_accuracy@10
            value: 0.9714285714285714
            name: Dot Accuracy@10
          - type: dot_precision@1
            value: 0.7214285714285714
            name: Dot Precision@1
          - type: dot_precision@3
            value: 0.2928571428571428
            name: Dot Precision@3
          - type: dot_precision@5
            value: 0.18999999999999995
            name: Dot Precision@5
          - type: dot_precision@10
            value: 0.09714285714285713
            name: Dot Precision@10
          - type: dot_recall@1
            value: 0.7214285714285714
            name: Dot Recall@1
          - type: dot_recall@3
            value: 0.8785714285714286
            name: Dot Recall@3
          - type: dot_recall@5
            value: 0.95
            name: Dot Recall@5
          - type: dot_recall@10
            value: 0.9714285714285714
            name: Dot Recall@10
          - type: dot_ndcg@10
            value: 0.8514639427234363
            name: Dot Ndcg@10
          - type: dot_mrr@10
            value: 0.8122108843537416
            name: Dot Mrr@10
          - type: dot_map@100
            value: 0.8142292826221397
            name: Dot Map@100

SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-MiniLM-L6-v2
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 tokens
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("yinong333/finetuned_MiniLM")
# Run inference
sentences = [
    'What legal action did the Federal Trade Commission take against Kochava regarding data tracking?',
    'ENDNOTES\n75. See., e.g., Sam Sabin. Digital surveillance in a post-Roe world. Politico. May 5, 2022. https://\nwww.politico.com/newsletters/digital-future-daily/2022/05/05/digital-surveillance-in-a-post-roe\xad\nworld-00030459; Federal Trade Commission. FTC Sues Kochava for Selling Data that Tracks People at\nReproductive Health Clinics, Places of Worship, and Other Sensitive Locations. Aug. 29, 2022. https://\nwww.ftc.gov/news-events/news/press-releases/2022/08/ftc-sues-kochava-selling-data-tracks-people\xad\nreproductive-health-clinics-places-worship-other\n76. Todd Feathers. This Private Equity Firm Is Amassing Companies That Collect Data on America’s\nChildren. The Markup. Jan. 11, 2022.\nhttps://themarkup.org/machine-learning/2022/01/11/this-private-equity-firm-is-amassing-companies\xad\nthat-collect-data-on-americas-children\n77. Reed Albergotti. Every employee who leaves Apple becomes an ‘associate’: In job databases used by',
    'DATA PRIVACY \nEXTRA PROTECTIONS FOR DATA RELATED TO SENSITIVE\nDOMAINS\n•\nContinuous positive airway pressure machines gather data for medical purposes, such as diagnosing sleep\napnea, and send usage data to a patient’s insurance company, which may subsequently deny coverage for the\ndevice based on usage data. Patients were not aware that the data would be used in this way or monitored\nby anyone other than their doctor.70 \n•\nA department store company used predictive analytics applied to collected consumer data to determine that a\nteenage girl was pregnant, and sent maternity clothing ads and other baby-related advertisements to her\nhouse, revealing to her father that she was pregnant.71\n•\nSchool audio surveillance systems monitor student conversations to detect potential "stress indicators" as\na warning of potential violence.72 Online proctoring systems claim to detect if a student is cheating on an',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.7214
cosine_accuracy@3	0.8786
cosine_accuracy@5	0.95
cosine_accuracy@10	0.9714
cosine_precision@1	0.7214
cosine_precision@3	0.2929
cosine_precision@5	0.19
cosine_precision@10	0.0971
cosine_recall@1	0.7214
cosine_recall@3	0.8786
cosine_recall@5	0.95
cosine_recall@10	0.9714
cosine_ndcg@10	0.8515
cosine_mrr@10	0.8122
cosine_map@100	0.8142
dot_accuracy@1	0.7214
dot_accuracy@3	0.8786
dot_accuracy@5	0.95
dot_accuracy@10	0.9714
dot_precision@1	0.7214
dot_precision@3	0.2929
dot_precision@5	0.19
dot_precision@10	0.0971
dot_recall@1	0.7214
dot_recall@3	0.8786
dot_recall@5	0.95
dot_recall@10	0.9714
dot_ndcg@10	0.8515
dot_mrr@10	0.8122
dot_map@100	0.8142

Training Details

Training Dataset

Unnamed Dataset

Size: 760 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 760 samples:
sentence_0 sentence_1
type string string
details
min: 11 tokens
mean: 20.96 tokens
max: 36 tokens

min: 21 tokens
mean: 167.91 tokens
max: 256 tokens

	sentence_0	sentence_1
type	string	string
details	min: 11 tokens mean: 20.96 tokens max: 36 tokens	min: 21 tokens mean: 167.91 tokens max: 256 tokens

Samples:

sentence_0	sentence_1
`What is the purpose of the AI Bill of Rights mentioned in the context?`	`BLUEPRINT FOR AN AI BILL OF RIGHTS MAKING AUTOMATED SYSTEMS WORK FOR THE AMERICAN PEOPLE OCTOBER 2022`
`When was the Blueprint for an AI Bill of Rights published?`	`BLUEPRINT FOR AN AI BILL OF RIGHTS MAKING AUTOMATED SYSTEMS WORK FOR THE AMERICAN PEOPLE OCTOBER 2022`
`What was the purpose of the Blueprint for an AI Bill of Rights published by the White House Office of Science and Technology Policy?`	About this Document The Blueprint for an AI Bill of Rights: Making Automated Systems Work for the American People was published by the White House Office of Science and Technology Policy in October 2022. This framework was released one year after OSTP announced the launch of a process to develop “a bill of rights for an AI-powered world.” Its release follows a year of public engagement to inform this initiative. The framework is available online at: https://www.whitehouse.gov/ostp/ai-bill-of-rights About the Office of Science and Technology Policy The Office of Science and Technology Policy (OSTP) was established by the National Science and Technology Policy, Organization, and Priorities Act of 1976 to provide the President and others within the Executive Office of the President with advice on the scientific, engineering, and technological aspects of the economy, national

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        384,
        128
    ],
    "matryoshka_weights": [
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 30
per_device_eval_batch_size: 30
num_train_epochs: 5
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 30
per_device_eval_batch_size: 30
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 5
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
eval_use_gather_object: False
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	cosine_map@100
1.0	26	0.7610
1.9231	50	0.8047
2.0	52	0.8051
3.0	78	0.8116
3.8462	100	0.8142

Framework Versions

Python: 3.10.12
Sentence Transformers: 3.1.1
Transformers: 4.44.2
PyTorch: 2.4.1+cu121
Accelerate: 0.34.2
Datasets: 2.19.2
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}