SetFit with BAAI/bge-small-en-v1.5
This is a SetFit model that can be used for Text Classification. This SetFit model uses BAAI/bge-small-en-v1.5 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: BAAI/bge-small-en-v1.5
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 512 tokens
- Number of Classes: 4 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
Label | Examples |
---|---|
2 |
|
1 |
|
0 |
|
3 |
|
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the ๐ค Hub
model = SetFitModel.from_pretrained("research-dump/bge-small-en-v1.5_wikipedia_gr_stance_prediction_en")
# Run inference
preds = model("Meets . &mdash")
Training Details
Training Set Metrics
Training set | Min | Median | Max |
---|---|---|---|
Word count | 2 | 35.91 | 244 |
Label | Training Sample Count |
---|---|
0 | 7 |
1 | 64 |
2 | 25 |
3 | 4 |
Training Hyperparameters
- batch_size: (4, 1)
- num_epochs: (5, 5)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 10
- body_learning_rate: (1e-05, 1e-05)
- head_learning_rate: 5e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: True
- use_amp: True
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
Training Results
Epoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.002 | 1 | 0.2329 | - |
1.0 | 500 | 0.166 | 0.2258 |
2.0 | 1000 | 0.02 | 0.2638 |
3.0 | 1500 | 0.0068 | 0.2447 |
4.0 | 2000 | 0.0042 | 0.2561 |
5.0 | 2500 | 0.0036 | 0.2562 |
Framework Versions
- Python: 3.10.12
- SetFit: 1.1.1
- Sentence Transformers: 3.3.1
- Transformers: 4.48.2
- PyTorch: 2.2.1+cu121
- Datasets: 2.21.0
- Tokenizers: 0.21.0
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
- Downloads last month
- 6
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
Model tree for research-dump/bge-small-en-v1.5_wikipedia_gr_stance_prediction_en
Base model
BAAI/bge-small-en-v1.5