|
--- |
|
language: |
|
- en |
|
license: apache-2.0 |
|
library_name: sentence-transformers |
|
tags: |
|
- sentence-transformers |
|
- sentence-similarity |
|
- feature-extraction |
|
- generated_from_trainer |
|
- dataset_size:6300 |
|
- loss:MatryoshkaLoss |
|
- loss:MultipleNegativesRankingLoss |
|
base_model: BAAI/bge-base-en-v1.5 |
|
datasets: [] |
|
metrics: |
|
- cosine_accuracy@1 |
|
- cosine_accuracy@3 |
|
- cosine_accuracy@5 |
|
- cosine_accuracy@10 |
|
- cosine_precision@1 |
|
- cosine_precision@3 |
|
- cosine_precision@5 |
|
- cosine_precision@10 |
|
- cosine_recall@1 |
|
- cosine_recall@3 |
|
- cosine_recall@5 |
|
- cosine_recall@10 |
|
- cosine_ndcg@10 |
|
- cosine_mrr@10 |
|
- cosine_map@100 |
|
widget: |
|
- source_sentence: 'Forward-looking statements may appear throughout this report, |
|
including without limitation, the following sections: “Management''s Discussion |
|
and Analysis,” “Risk Factors” and "Notes 4, 8 and 13 to the Consolidated Financial |
|
Statements."' |
|
sentences: |
|
- How does a one-year adjustment in the 2023 expected retirement age for U.S. plans |
|
affect income before income taxes? |
|
- Which sections of the report might contain forward-looking statements according |
|
to the text? |
|
- What was the allowance for loan and lease losses at Bank of America as of December |
|
31, 2022? |
|
- source_sentence: Interest income | $ | 267 | | | $ | 29 | | $ | 238 | | 821 | % |
|
sentences: |
|
- What are the key risks and uncertainties mentioned that could impact the validity |
|
of DaVita's forward-looking statements? |
|
- How did the interest income change in fiscal year 2023 compared to the previous |
|
year? |
|
- What are some of the main competitive factors in the interactive entertainment |
|
industry? |
|
- source_sentence: Veklury received U.S. Food and Drug Administration (FDA) and European |
|
Commission (EC) approval to treat COVID-19 in patients with mild to severe hepatic |
|
impairment and those with severe renal impairment, including those on dialysis. |
|
sentences: |
|
- What significant regulatory approvals did Gilead's Veklury receive? |
|
- What type of information is included under the caption "Legal Proceedings" in |
|
an Annual Report on Form 10-K? |
|
- What was the cash change related to changes in operating assets and liabilities, |
|
including working capital, in 2022? |
|
- source_sentence: The net value of property, plant, and equipment for the consolidated |
|
group increased from $12,028 million in 2022 to $12,680 million in 2023. |
|
sentences: |
|
- What steps does the company plan to take next after discussing data with regulators |
|
and key opinion leaders? |
|
- How does the company manage fluctuations in foreign currency exchange rates? |
|
- What was the increase in property, plant, and equipment net value from 2022 to |
|
2023 for the consolidated group? |
|
- source_sentence: The effective duration of our total AFS and HTM investments securities |
|
as of December 31, 2023 is approximately 3.9 years. |
|
sentences: |
|
- What are the effective durations of the total Available-for-Sale (AFS) and Held-to-Maturity |
|
(HTM) investment securities as of December 31, 2023? |
|
- What was the net unit growth percentage for Hilton in the year ended December |
|
31, 2023? |
|
- What does goodwill represent in accounting? |
|
pipeline_tag: sentence-similarity |
|
model-index: |
|
- name: BGE base Financial Matryoshka |
|
results: |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 768 |
|
type: dim_768 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.7285714285714285 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8485714285714285 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8885714285714286 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9214285714285714 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.7285714285714285 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.28285714285714286 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.17771428571428569 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09214285714285712 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.7285714285714285 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8485714285714285 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8885714285714286 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9214285714285714 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8274202252845575 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7969903628117911 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7998523047098398 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 512 |
|
type: dim_512 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.72 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8442857142857143 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8785714285714286 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.92 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.72 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.2814285714285714 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.17571428571428568 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09199999999999998 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.72 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8442857142857143 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8785714285714286 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.92 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8213589464095679 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7896825396825394 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7926726035572866 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 256 |
|
type: dim_256 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.7214285714285714 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8385714285714285 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8742857142857143 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9128571428571428 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.7214285714285714 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.27952380952380956 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.17485714285714282 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09128571428571428 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.7214285714285714 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8385714285714285 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8742857142857143 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9128571428571428 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8190844047519252 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7888673469387758 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7921199469128796 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 128 |
|
type: dim_128 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.6971428571428572 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.8328571428571429 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8671428571428571 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.9057142857142857 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.6971428571428572 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.2776190476190476 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.1734285714285714 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.09057142857142855 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.6971428571428572 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.8328571428571429 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8671428571428571 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.9057142857142857 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.8054254319689889 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.7729421768707481 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.776216648701894 |
|
name: Cosine Map@100 |
|
- task: |
|
type: information-retrieval |
|
name: Information Retrieval |
|
dataset: |
|
name: dim 64 |
|
type: dim_64 |
|
metrics: |
|
- type: cosine_accuracy@1 |
|
value: 0.6614285714285715 |
|
name: Cosine Accuracy@1 |
|
- type: cosine_accuracy@3 |
|
value: 0.7985714285714286 |
|
name: Cosine Accuracy@3 |
|
- type: cosine_accuracy@5 |
|
value: 0.8442857142857143 |
|
name: Cosine Accuracy@5 |
|
- type: cosine_accuracy@10 |
|
value: 0.8814285714285715 |
|
name: Cosine Accuracy@10 |
|
- type: cosine_precision@1 |
|
value: 0.6614285714285715 |
|
name: Cosine Precision@1 |
|
- type: cosine_precision@3 |
|
value: 0.26619047619047614 |
|
name: Cosine Precision@3 |
|
- type: cosine_precision@5 |
|
value: 0.16885714285714284 |
|
name: Cosine Precision@5 |
|
- type: cosine_precision@10 |
|
value: 0.08814285714285712 |
|
name: Cosine Precision@10 |
|
- type: cosine_recall@1 |
|
value: 0.6614285714285715 |
|
name: Cosine Recall@1 |
|
- type: cosine_recall@3 |
|
value: 0.7985714285714286 |
|
name: Cosine Recall@3 |
|
- type: cosine_recall@5 |
|
value: 0.8442857142857143 |
|
name: Cosine Recall@5 |
|
- type: cosine_recall@10 |
|
value: 0.8814285714285715 |
|
name: Cosine Recall@10 |
|
- type: cosine_ndcg@10 |
|
value: 0.7728992637054746 |
|
name: Cosine Ndcg@10 |
|
- type: cosine_mrr@10 |
|
value: 0.737815759637188 |
|
name: Cosine Mrr@10 |
|
- type: cosine_map@100 |
|
value: 0.7417951294330247 |
|
name: Cosine Map@100 |
|
--- |
|
|
|
# BGE base Financial Matryoshka |
|
|
|
This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. |
|
|
|
## Model Details |
|
|
|
### Model Description |
|
- **Model Type:** Sentence Transformer |
|
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a --> |
|
- **Maximum Sequence Length:** 512 tokens |
|
- **Output Dimensionality:** 768 tokens |
|
- **Similarity Function:** Cosine Similarity |
|
<!-- - **Training Dataset:** Unknown --> |
|
- **Language:** en |
|
- **License:** apache-2.0 |
|
|
|
### Model Sources |
|
|
|
- **Documentation:** [Sentence Transformers Documentation](https://sbert.net) |
|
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) |
|
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) |
|
|
|
### Full Model Architecture |
|
|
|
``` |
|
SentenceTransformer( |
|
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel |
|
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) |
|
(2): Normalize() |
|
) |
|
``` |
|
|
|
## Usage |
|
|
|
### Direct Usage (Sentence Transformers) |
|
|
|
First install the Sentence Transformers library: |
|
|
|
```bash |
|
pip install -U sentence-transformers |
|
``` |
|
|
|
Then you can load this model and run inference. |
|
```python |
|
from sentence_transformers import SentenceTransformer |
|
|
|
# Download from the 🤗 Hub |
|
model = SentenceTransformer("Hritikmore/bge-base-financial-matryoshka") |
|
# Run inference |
|
sentences = [ |
|
'The effective duration of our total AFS and HTM investments securities as of December 31, 2023 is approximately 3.9 years.', |
|
'What are the effective durations of the total Available-for-Sale (AFS) and Held-to-Maturity (HTM) investment securities as of December 31, 2023?', |
|
'What was the net unit growth percentage for Hilton in the year ended December 31, 2023?', |
|
] |
|
embeddings = model.encode(sentences) |
|
print(embeddings.shape) |
|
# [3, 768] |
|
|
|
# Get the similarity scores for the embeddings |
|
similarities = model.similarity(embeddings, embeddings) |
|
print(similarities.shape) |
|
# [3, 3] |
|
``` |
|
|
|
<!-- |
|
### Direct Usage (Transformers) |
|
|
|
<details><summary>Click to see the direct usage in Transformers</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Downstream Usage (Sentence Transformers) |
|
|
|
You can finetune this model on your own dataset. |
|
|
|
<details><summary>Click to expand</summary> |
|
|
|
</details> |
|
--> |
|
|
|
<!-- |
|
### Out-of-Scope Use |
|
|
|
*List how the model may foreseeably be misused and address what users ought not to do with the model.* |
|
--> |
|
|
|
## Evaluation |
|
|
|
### Metrics |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_768` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.7286 | |
|
| cosine_accuracy@3 | 0.8486 | |
|
| cosine_accuracy@5 | 0.8886 | |
|
| cosine_accuracy@10 | 0.9214 | |
|
| cosine_precision@1 | 0.7286 | |
|
| cosine_precision@3 | 0.2829 | |
|
| cosine_precision@5 | 0.1777 | |
|
| cosine_precision@10 | 0.0921 | |
|
| cosine_recall@1 | 0.7286 | |
|
| cosine_recall@3 | 0.8486 | |
|
| cosine_recall@5 | 0.8886 | |
|
| cosine_recall@10 | 0.9214 | |
|
| cosine_ndcg@10 | 0.8274 | |
|
| cosine_mrr@10 | 0.797 | |
|
| **cosine_map@100** | **0.7999** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_512` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.72 | |
|
| cosine_accuracy@3 | 0.8443 | |
|
| cosine_accuracy@5 | 0.8786 | |
|
| cosine_accuracy@10 | 0.92 | |
|
| cosine_precision@1 | 0.72 | |
|
| cosine_precision@3 | 0.2814 | |
|
| cosine_precision@5 | 0.1757 | |
|
| cosine_precision@10 | 0.092 | |
|
| cosine_recall@1 | 0.72 | |
|
| cosine_recall@3 | 0.8443 | |
|
| cosine_recall@5 | 0.8786 | |
|
| cosine_recall@10 | 0.92 | |
|
| cosine_ndcg@10 | 0.8214 | |
|
| cosine_mrr@10 | 0.7897 | |
|
| **cosine_map@100** | **0.7927** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_256` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.7214 | |
|
| cosine_accuracy@3 | 0.8386 | |
|
| cosine_accuracy@5 | 0.8743 | |
|
| cosine_accuracy@10 | 0.9129 | |
|
| cosine_precision@1 | 0.7214 | |
|
| cosine_precision@3 | 0.2795 | |
|
| cosine_precision@5 | 0.1749 | |
|
| cosine_precision@10 | 0.0913 | |
|
| cosine_recall@1 | 0.7214 | |
|
| cosine_recall@3 | 0.8386 | |
|
| cosine_recall@5 | 0.8743 | |
|
| cosine_recall@10 | 0.9129 | |
|
| cosine_ndcg@10 | 0.8191 | |
|
| cosine_mrr@10 | 0.7889 | |
|
| **cosine_map@100** | **0.7921** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_128` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.6971 | |
|
| cosine_accuracy@3 | 0.8329 | |
|
| cosine_accuracy@5 | 0.8671 | |
|
| cosine_accuracy@10 | 0.9057 | |
|
| cosine_precision@1 | 0.6971 | |
|
| cosine_precision@3 | 0.2776 | |
|
| cosine_precision@5 | 0.1734 | |
|
| cosine_precision@10 | 0.0906 | |
|
| cosine_recall@1 | 0.6971 | |
|
| cosine_recall@3 | 0.8329 | |
|
| cosine_recall@5 | 0.8671 | |
|
| cosine_recall@10 | 0.9057 | |
|
| cosine_ndcg@10 | 0.8054 | |
|
| cosine_mrr@10 | 0.7729 | |
|
| **cosine_map@100** | **0.7762** | |
|
|
|
#### Information Retrieval |
|
* Dataset: `dim_64` |
|
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) |
|
|
|
| Metric | Value | |
|
|:--------------------|:-----------| |
|
| cosine_accuracy@1 | 0.6614 | |
|
| cosine_accuracy@3 | 0.7986 | |
|
| cosine_accuracy@5 | 0.8443 | |
|
| cosine_accuracy@10 | 0.8814 | |
|
| cosine_precision@1 | 0.6614 | |
|
| cosine_precision@3 | 0.2662 | |
|
| cosine_precision@5 | 0.1689 | |
|
| cosine_precision@10 | 0.0881 | |
|
| cosine_recall@1 | 0.6614 | |
|
| cosine_recall@3 | 0.7986 | |
|
| cosine_recall@5 | 0.8443 | |
|
| cosine_recall@10 | 0.8814 | |
|
| cosine_ndcg@10 | 0.7729 | |
|
| cosine_mrr@10 | 0.7378 | |
|
| **cosine_map@100** | **0.7418** | |
|
|
|
<!-- |
|
## Bias, Risks and Limitations |
|
|
|
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.* |
|
--> |
|
|
|
<!-- |
|
### Recommendations |
|
|
|
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.* |
|
--> |
|
|
|
## Training Details |
|
|
|
### Training Dataset |
|
|
|
#### Unnamed Dataset |
|
|
|
|
|
* Size: 6,300 training samples |
|
* Columns: <code>positive</code> and <code>anchor</code> |
|
* Approximate statistics based on the first 1000 samples: |
|
| | positive | anchor | |
|
|:--------|:-----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| |
|
| type | string | string | |
|
| details | <ul><li>min: 2 tokens</li><li>mean: 45.87 tokens</li><li>max: 272 tokens</li></ul> | <ul><li>min: 2 tokens</li><li>mean: 20.43 tokens</li><li>max: 41 tokens</li></ul> | |
|
* Samples: |
|
| positive | anchor | |
|
|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------| |
|
| <code>Significant judgment is required in evaluating our tax positions and during the ordinary course of business, there are many transactions and calculations for which the ultimate tax settlement is uncertain. As a result, we recognize the effect of this uncertainty on our tax attributes or taxes payable based on our estimates of the eventual outcome.</code> | <code>Why might the company's tax settlements vary?</code> | |
|
| <code>OPSUMIT is used for the treatment of pediatric pulmonary arterial hypertension.</code> | <code>What medical condition does OPSUMIT treat?</code> | |
|
| <code>Tangible equity ratios and tangible book value per share of common stock are non-GAAP financial measures. For more information on these ratios and corresponding reconciliations to GAAP financial measures, see Supplemental Financial Data and Non-GAAP Reconciliations.</code> | <code>What is the tangible equity ratio considered according to standard financial measures?</code> | |
|
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: |
|
```json |
|
{ |
|
"loss": "MultipleNegativesRankingLoss", |
|
"matryoshka_dims": [ |
|
768, |
|
512, |
|
256, |
|
128, |
|
64 |
|
], |
|
"matryoshka_weights": [ |
|
1, |
|
1, |
|
1, |
|
1, |
|
1 |
|
], |
|
"n_dims_per_step": -1 |
|
} |
|
``` |
|
|
|
### Training Hyperparameters |
|
#### Non-Default Hyperparameters |
|
|
|
- `eval_strategy`: epoch |
|
- `gradient_accumulation_steps`: 16 |
|
- `learning_rate`: 2e-05 |
|
- `num_train_epochs`: 2 |
|
- `lr_scheduler_type`: cosine |
|
- `warmup_ratio`: 0.1 |
|
- `tf32`: False |
|
- `load_best_model_at_end`: True |
|
- `optim`: adamw_torch_fused |
|
- `batch_sampler`: no_duplicates |
|
|
|
#### All Hyperparameters |
|
<details><summary>Click to expand</summary> |
|
|
|
- `overwrite_output_dir`: False |
|
- `do_predict`: False |
|
- `eval_strategy`: epoch |
|
- `prediction_loss_only`: True |
|
- `per_device_train_batch_size`: 8 |
|
- `per_device_eval_batch_size`: 8 |
|
- `per_gpu_train_batch_size`: None |
|
- `per_gpu_eval_batch_size`: None |
|
- `gradient_accumulation_steps`: 16 |
|
- `eval_accumulation_steps`: None |
|
- `learning_rate`: 2e-05 |
|
- `weight_decay`: 0.0 |
|
- `adam_beta1`: 0.9 |
|
- `adam_beta2`: 0.999 |
|
- `adam_epsilon`: 1e-08 |
|
- `max_grad_norm`: 1.0 |
|
- `num_train_epochs`: 2 |
|
- `max_steps`: -1 |
|
- `lr_scheduler_type`: cosine |
|
- `lr_scheduler_kwargs`: {} |
|
- `warmup_ratio`: 0.1 |
|
- `warmup_steps`: 0 |
|
- `log_level`: passive |
|
- `log_level_replica`: warning |
|
- `log_on_each_node`: True |
|
- `logging_nan_inf_filter`: True |
|
- `save_safetensors`: True |
|
- `save_on_each_node`: False |
|
- `save_only_model`: False |
|
- `restore_callback_states_from_checkpoint`: False |
|
- `no_cuda`: False |
|
- `use_cpu`: False |
|
- `use_mps_device`: False |
|
- `seed`: 42 |
|
- `data_seed`: None |
|
- `jit_mode_eval`: False |
|
- `use_ipex`: False |
|
- `bf16`: False |
|
- `fp16`: False |
|
- `fp16_opt_level`: O1 |
|
- `half_precision_backend`: auto |
|
- `bf16_full_eval`: False |
|
- `fp16_full_eval`: False |
|
- `tf32`: False |
|
- `local_rank`: 0 |
|
- `ddp_backend`: None |
|
- `tpu_num_cores`: None |
|
- `tpu_metrics_debug`: False |
|
- `debug`: [] |
|
- `dataloader_drop_last`: False |
|
- `dataloader_num_workers`: 0 |
|
- `dataloader_prefetch_factor`: None |
|
- `past_index`: -1 |
|
- `disable_tqdm`: False |
|
- `remove_unused_columns`: True |
|
- `label_names`: None |
|
- `load_best_model_at_end`: True |
|
- `ignore_data_skip`: False |
|
- `fsdp`: [] |
|
- `fsdp_min_num_params`: 0 |
|
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} |
|
- `fsdp_transformer_layer_cls_to_wrap`: None |
|
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} |
|
- `deepspeed`: None |
|
- `label_smoothing_factor`: 0.0 |
|
- `optim`: adamw_torch_fused |
|
- `optim_args`: None |
|
- `adafactor`: False |
|
- `group_by_length`: False |
|
- `length_column_name`: length |
|
- `ddp_find_unused_parameters`: None |
|
- `ddp_bucket_cap_mb`: None |
|
- `ddp_broadcast_buffers`: False |
|
- `dataloader_pin_memory`: True |
|
- `dataloader_persistent_workers`: False |
|
- `skip_memory_metrics`: True |
|
- `use_legacy_prediction_loop`: False |
|
- `push_to_hub`: False |
|
- `resume_from_checkpoint`: None |
|
- `hub_model_id`: None |
|
- `hub_strategy`: every_save |
|
- `hub_private_repo`: False |
|
- `hub_always_push`: False |
|
- `gradient_checkpointing`: False |
|
- `gradient_checkpointing_kwargs`: None |
|
- `include_inputs_for_metrics`: False |
|
- `eval_do_concat_batches`: True |
|
- `fp16_backend`: auto |
|
- `push_to_hub_model_id`: None |
|
- `push_to_hub_organization`: None |
|
- `mp_parameters`: |
|
- `auto_find_batch_size`: False |
|
- `full_determinism`: False |
|
- `torchdynamo`: None |
|
- `ray_scope`: last |
|
- `ddp_timeout`: 1800 |
|
- `torch_compile`: False |
|
- `torch_compile_backend`: None |
|
- `torch_compile_mode`: None |
|
- `dispatch_batches`: None |
|
- `split_batches`: None |
|
- `include_tokens_per_second`: False |
|
- `include_num_input_tokens_seen`: False |
|
- `neftune_noise_alpha`: None |
|
- `optim_target_modules`: None |
|
- `batch_eval_metrics`: False |
|
- `batch_sampler`: no_duplicates |
|
- `multi_dataset_batch_sampler`: proportional |
|
|
|
</details> |
|
|
|
### Training Logs |
|
| Epoch | Step | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 | |
|
|:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|:----------------------:| |
|
| 0.2030 | 10 | 0.7168 | - | - | - | - | - | |
|
| 0.4061 | 20 | 0.3345 | - | - | - | - | - | |
|
| 0.6091 | 30 | 0.2234 | - | - | - | - | - | |
|
| 0.8122 | 40 | 0.2126 | - | - | - | - | - | |
|
| **0.9949** | **49** | **-** | **0.7796** | **0.7844** | **0.7905** | **0.7293** | **0.7973** | |
|
| 1.0152 | 50 | 0.2301 | - | - | - | - | - | |
|
| 1.2183 | 60 | 0.1595 | - | - | - | - | - | |
|
| 1.4213 | 70 | 0.1082 | - | - | - | - | - | |
|
| 1.6244 | 80 | 0.0911 | - | - | - | - | - | |
|
| 1.8274 | 90 | 0.1068 | - | - | - | - | - | |
|
| 1.9898 | 98 | - | 0.7762 | 0.7921 | 0.7927 | 0.7418 | 0.7999 | |
|
|
|
* The bold row denotes the saved checkpoint. |
|
|
|
### Framework Versions |
|
- Python: 3.10.12 |
|
- Sentence Transformers: 3.0.1 |
|
- Transformers: 4.41.2 |
|
- PyTorch: 2.1.2+cu121 |
|
- Accelerate: 0.31.0 |
|
- Datasets: 2.19.1 |
|
- Tokenizers: 0.19.1 |
|
|
|
## Citation |
|
|
|
### BibTeX |
|
|
|
#### Sentence Transformers |
|
```bibtex |
|
@inproceedings{reimers-2019-sentence-bert, |
|
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", |
|
author = "Reimers, Nils and Gurevych, Iryna", |
|
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", |
|
month = "11", |
|
year = "2019", |
|
publisher = "Association for Computational Linguistics", |
|
url = "https://arxiv.org/abs/1908.10084", |
|
} |
|
``` |
|
|
|
#### MatryoshkaLoss |
|
```bibtex |
|
@misc{kusupati2024matryoshka, |
|
title={Matryoshka Representation Learning}, |
|
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, |
|
year={2024}, |
|
eprint={2205.13147}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.LG} |
|
} |
|
``` |
|
|
|
#### MultipleNegativesRankingLoss |
|
```bibtex |
|
@misc{henderson2017efficient, |
|
title={Efficient Natural Language Response Suggestion for Smart Reply}, |
|
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, |
|
year={2017}, |
|
eprint={1705.00652}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
|
|
|
<!-- |
|
## Glossary |
|
|
|
*Clearly define terms in order to be accessible across audiences.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Authors |
|
|
|
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.* |
|
--> |
|
|
|
<!-- |
|
## Model Card Contact |
|
|
|
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.* |
|
--> |
