---
base_model: basel/ATTACK-BERT
library_name: setfit
metrics:
- accuracy
pipeline_tag: text-classification
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: With the exception of the DHSK and the RNG seed, all critical security parameters
    are loaded during manufacturing.
- text: The private key component of an ANSI X9.31-compliant PRNG is stored securely
    in NVRAM.
- text: This DRNG uses an 8-byte Seed and an 16-byte Seed Key as inputs to the DRNG.
    The seed & seed-key values are generated by the hardware RNG and stored only in
    RAM. These values are zeroized when the module is reset in contact mode or when
    the module is deselected in contactless mode.
- text: The seed key is used as an input to the X9.31 RNG, a deterministic random
    number generator, and is generally not stored long term.
- text: PRNG seed key X9.31 SDRAM This is the seed key for the PRNG. It is statically
    stored in the code.
inference: true
---

# SetFit with basel/ATTACK-BERT

This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Text Classification. This SetFit model uses [basel/ATTACK-BERT](https://huggingface.co/basel/ATTACK-BERT) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

## Model Details

### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [basel/ATTACK-BERT](https://huggingface.co/basel/ATTACK-BERT)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 384 tokens
- **Number of Classes:** 2 classes
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)

### Model Labels
| Label    | Examples                                                                                                                                                                                                                                                                                                                                                                                                                                |
|:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| negative | <ul><li>'ANSI X9.31 Appendix A.2.4 PRNG key AES 128-bit key Internally generated Never exits the module Plaintext in volatile memory Rebooting the modules Seeding the FIPS-Approved ANSI X9.31 PRNG'</li><li>'PRNG seed key Continually polled from various system resources to accrue entropy.'</li><li>'module stores RNG and DRBG state values only in RAM.'</li></ul>                                                              |
| positive | <ul><li>"The PRNG's seed key is encrypted with a device-specific key and securely stored in non-volatile memory."</li><li>'An RNG key compliant with ANSI X9.31 AES 128-bit standards is used by the underlying encryption algorithm and stored in plaintext within tamper-protected memory during factory setup.'</li><li>'The PRNG seed key is pre-loaded during manufacturing and compiled directly into the binary code.'</li></ul> |

## Uses

### Direct Use for Inference

First install the SetFit library:

```bash
pip install setfit
```

Then you can load this model and run inference.

```python
from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("yasirdemircan/setfit_rng_v3")
# Run inference
preds = model("The private key component of an ANSI X9.31-compliant PRNG is stored securely in NVRAM.")
```

<!--
### Downstream Use

*List how someone could finetune this model on their own dataset.*
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Set Metrics
| Training set | Min | Median  | Max |
|:-------------|:----|:--------|:----|
| Word count   | 6   | 18.8444 | 59  |

| Label    | Training Sample Count |
|:---------|:----------------------|
| negative | 21                    |
| positive | 24                    |

### Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (4, 4)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True

### Training Results
| Epoch  | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0149 | 1    | 0.2442        | -               |
| 0.7463 | 50   | 0.1714        | -               |
| 1.0    | 67   | -             | 0.1785          |
| 1.4925 | 100  | 0.0029        | -               |
| 2.0    | 134  | -             | 0.1880          |
| 2.2388 | 150  | 0.0004        | -               |
| 2.9851 | 200  | 0.0003        | -               |
| 3.0    | 201  | -             | 0.1818          |
| 3.7313 | 250  | 0.0003        | -               |
| 4.0    | 268  | -             | 0.1837          |

### Framework Versions
- Python: 3.10.15
- SetFit: 1.2.0.dev0
- Sentence Transformers: 3.3.1
- Transformers: 4.45.2
- PyTorch: 2.5.1+cu124
- Datasets: 2.19.1
- Tokenizers: 0.20.1

## Citation

### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->