---
library_name: transformers
license: mit
base_model: intfloat/e5-base
tags:
- generated_from_trainer
model-index:
- name: e5-base-german-sentence-similarity
  results: []
datasets:
- PhilipMay/stsb_multi_mt
language:
- de
metrics:
- pearsonr
pipeline_tag: sentence-similarity
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# e5-base-german-sentence-similarity

This model is a fine-tuned version of [intfloat/e5-base](https://huggingface.co/intfloat/e5-base) on german subset of the stsb_multi_mt dataset.
It achieves the following results on the evaluation set:

Validation:
- Loss: 0.9118
- Pearson:  0.7952
Test:
- Loss: 1.2162
- Pearson: 0.7252

## Usage

```python
#install sentence-transformers
!pip install -U sentence-transformers

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("kaixkhazaki/e5-base-german-sentence-similarity")

sentences = [
    'Ein älterer Herr genießt die Natur auf einer Parkbank.',
    'Ein alter Mann vertieft sich in eine Zeitung im Stadtpark.',
    'Ein Teenager hört Musik auf einer Bank.'
]
embeddings = model.encode(sentences)

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)

>>
tensor([[1.0000, 0.9044, 0.7919],
        [0.9044, 1.0000, 0.7757],
        [0.7919, 0.7757, 1.0000]])


```

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

- **Dataset:** [STS Benchmark](https://huggingface.co/datasets/stsb_multi_mt)
- 
## Training procedure

 additional dropout implemented on model to avoid overfitting(hidden_dropout_prob=0.25 ,attention_probs_dropout_prob = 0.25)

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 1e-05
- train_batch_size: 16
- eval_batch_size: 32
- seed: 42
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 500
- num_epochs: 12

### Training results

| Training Loss | Epoch | Step | Validation Loss | Pearson |
|:-------------:|:-----:|:----:|:---------------:|:-------:|
| 1.6556        | 1.0   | 360  | 1.3086          | 0.7176  |
| 1.0113        | 2.0   | 720  | 1.2079          | 0.7563  |
| 1.0205        | 3.0   | 1080 | 1.2091          | 0.7639  |
| 0.8876        | 4.0   | 1440 | 0.9325          | 0.7910  |
| 0.6762        | 5.0   | 1800 | 0.9118          | 0.7952  |
| 0.5615        | 6.0   | 2160 | 0.9772          | 0.7910  |
| 0.6387        | 7.0   | 2520 | 0.9085          | 0.7933  |


### Framework versions

- Transformers 4.48.0
- Pytorch 2.2.2
- Datasets 3.2.0
- Tokenizers 0.21.0