README.md

---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:19979
- loss:CoSENTLoss
base_model: avsolatorio/GIST-small-Embedding-v0
widget:
- source_sentence: why oval face shape is attractive?
  sentences:
  - Both are also designed to add depth to your face, but in different ways. Bronzing
    primarily warms up your face, adding color in places where the sun would naturally
    hit. ... On the other hand, contouring is a makeup artist-approved technique that
    works to add structure and shadow to your face.
  - '''Ahjussi'' literally means uncle. You can use it for people who are very old
    than you. Like someone who is the age of your father, or someone with an age gap
    of 20 years or above.'
  - Most major banks will be open on Christmas Eve 2018, even though it's a federal
    holiday and generally recognized as a bank holiday.
- source_sentence: is ceo same as owner?
  sentences:
  - The CEO reports to the Chairman (acting on behalf of the Board) and to the Board
    directly. The Chairman is not responsible for executive matters regarding the
    Company's business. Other than the CEO and the Company Secretary, no executive
    reports to the Chairman, other than through the Board.
  - Understanding Deregulation In response to the country's greatest financial crisis
    in its history, Franklin D. Roosevelt's administration enacted many forms of financial
    regulation, including the Securities Exchange Acts of 1933 and 1934 and the U.S.
    Banking Act of 1933, otherwise known as the Glass-Steagall Act.
  - Gdzie kupić wodorosty? Na naszym rynku są głównie algi suszone. Bez problemu kupimy
    je w supermarketach, sklepach ze zdrową żywnością, bywają w rybnych - największy
    wybór jest oczywiście w sklepach internetowych. Za 10 arkuszy glonów nori zapłacimy
    ok.
- source_sentence: is gern stock a good buy?
  sentences:
  - 'The majority of these pads are made from one of two absorptive materials: Silica
    gel (a purified sand) or cellulose (a purified plant fiber), which are then coated
    in a non-toxic plastic wrapping that''s perforated, allowing the liquid to seep
    in and stay there.'
  - '[''The Vanguard Total Stock Market ETF (NYSEMKT:VTI)'', ''The Vanguard Total
    International Stock ETF (NASDAQ:VXUS)'', ''Amazon.com (NASDAQ:AMZN)'', ''Alphabet
    (NASDAQ:GOOG)(NASDAQ:GOOGL)'', ''Facebook (NASDAQ:FB)'', ''Intuitive Surgical
    (NASDAQ:ISRG)'']'
  - SCD is a disease that worsens over time. Treatments are available that can prevent
    complications and lengthen the lives of those who have this condition.
- source_sentence: where are sulfhydryl groups found?
  sentences:
  - Sulfhydryl groups can be found in the amino acid cysteine. When two cysteine residues
    are in close proximity to each other, they can form a disulfide bridge also called
    cystine.
  - '["On your Android phone or tablet, open your device''s Settings app .", ''Tap
    Google. Manage your Google Account.'', ''At the top, tap Personal info.'', ''Under
    "Profile," tap Name Edit. . You may be asked to sign in.'', ''Enter your name,
    then tap Done.'']'
  - '[''Difficulty digesting fat. It may take your body time to adjust to its new
    method of digesting fat. ... '', ''Diarrhea and flatulence. Indigestion can cause
    diarrhea or flatulence, often made worse by excess fat or too little fiber in
    the diet. ... '', ''Constipation. ... '', ''Intestinal injury. ... '', ''Jaundice
    or fever.'']'
- source_sentence: do assets in an irrevocable trust get a step up in basis?
  sentences:
  - An irrevocable grantor trust can own S corporation stock if it meets IRS regulations.
    ... If the trust owner designation is not made or is unclear, the trust will not
    qualify under IRS regulations. An irrevocable grantor trust does not have to make
    an election to be an S corporation shareholder.
  - Pineapple juice also contains bromelain, a group of enzymes linked to health benefits,
    such as reduced inflammation, improved digestion, and stronger immunity ( 9 ).
    Pineapple juice is rich in antioxidants, which help protect your body from damage
    and disease.
  - 'Ideally, fuel up two hours before you exercise by: Hydrating with water. Eating
    healthy carbohydrates such as whole-grain cereals (with low-fat or skim milk),
    whole-wheat toast, low-fat or fat-free yogurt, whole grain pasta, brown rice,
    fruits and vegetables.'
pipeline_tag: sentence-similarity
library_name: sentence-transformers
---

# SentenceTransformer based on avsolatorio/GIST-small-Embedding-v0

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [avsolatorio/GIST-small-Embedding-v0](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0) <!-- at revision 75e62fd210b9fde790430e0b2f040b0b00a021b1 -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 384 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("moshew/gist_small_ft_gooaq_v4")
# Run inference
sentences = [
    'do assets in an irrevocable trust get a step up in basis?',
    'An irrevocable grantor trust can own S corporation stock if it meets IRS regulations. ... If the trust owner designation is not made or is unclear, the trust will not qualify under IRS regulations. An irrevocable grantor trust does not have to make an election to be an S corporation shareholder.',
    'Ideally, fuel up two hours before you exercise by: Hydrating with water. Eating healthy carbohydrates such as whole-grain cereals (with low-fat or skim milk), whole-wheat toast, low-fat or fat-free yogurt, whole grain pasta, brown rice, fruits and vegetables.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### Unnamed Dataset

* Size: 19,979 training samples
* Columns: <code>sentence1</code>, <code>sentence2</code>, and <code>label</code>
* Approximate statistics based on the first 1000 samples:
  |         | sentence1                                                                         | sentence2                                                                           | label                                                         |
  |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:--------------------------------------------------------------|
  | type    | string                                                                            | string                                                                              | float                                                         |
  | details | <ul><li>min: 8 tokens</li><li>mean: 11.87 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 15 tokens</li><li>mean: 59.82 tokens</li><li>max: 139 tokens</li></ul> | <ul><li>min: 0.0</li><li>mean: 0.5</li><li>max: 1.0</li></ul> |
* Samples:
  | sentence1                                                          | sentence2                                                                                                                                                                                                                                                                                                                           | label            |
  |:-------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------|
  | <code>if someone blocked you on iphone can you send a text?</code> | <code>If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app.</code>                                                                        | <code>1.0</code> |
  | <code>if someone blocked you on iphone can you send a text?</code> | <code>If someone has blocked you on their device, you won't get an alert when it happens. You can still use iMessage to text your former contact, but they'll never receive the message or any notification of a text received in their Messages app. There is one clue that you've been blocked, though.</code>                    | <code>0.0</code> |
  | <code>can you have a relationship without expectations?</code>     | <code>Loving without expectations means being able to love someone even when they are letting you down. It means loving even when it feels awful. Even when you're crying so hard you can't see straight or say clear sentences. Loving someone without expectations means knowing they aren't perfect, but neither are you.</code> | <code>1.0</code> |
* Loss: [<code>CoSENTLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#cosentloss) with these parameters:
  ```json
  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `num_train_epochs`: 1
- `warmup_ratio`: 0.1
- `seed`: 12
- `bf16`: True
- `dataloader_num_workers`: 4

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: no
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 16
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 1
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 12
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 4
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `tp_size`: 0
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch  | Step | Training Loss |
|:------:|:----:|:-------------:|
| 0.0008 | 1    | 3.6013        |
| 0.8006 | 1000 | 3.4341        |


### Framework Versions
- Python: 3.11.12
- Sentence Transformers: 4.1.0
- Transformers: 4.51.3
- PyTorch: 2.6.0+cu124
- Accelerate: 1.5.2
- Datasets: 3.5.0
- Tokenizers: 0.21.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### CoSENTLoss
```bibtex
@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->